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ABSTRACT 


The purpose of this study was to determine if the usage of Thinking Maps® influenced 
the achievement scores of middle grades students in science on the Georgia Criterion-Referenced 
Competency Tests (CRCT) over a period of two years. Science test scores for the 2012-2013 
and 2013-2014 school years were analyzed by grade level and usage in SPSS and SAS, and after 
transformations were completed, non-parametric ANOVAs were conducted based on non-normal 
distributions. Science teachers were surveyed regarding their perceptions and usage of Thinking 
Maps®. The scores and surveys were then analyzed for simultaneous effects. 

The results of the study for the 2012-2013 data indicated grade level was significant and 
did influence a variation in scores, with seventh grade having a greater frequency of scores above 
850. Usage of Thinking Maps® was not significant and did not influence a variation in scores. 
Friedman’s non-parametric ANOVA showed 9.11% of the variance in scores was influenced by 
grade level and usage. The 2013-2014 data indicated grade level was significant and did 
influence a variation in scores, with seventh grade having a greater frequency of scores above 
825. Usage of Thinking Maps® was significant and did influence a variation in scores. 
Friedman’s non-parametric ANOVA revealed 23.2% of the variance in scores was influenced by 
grade level and usage. An interaction effect for either year could not be determined because of 
non-normal distributions and sample origination differences. Therefore, a comparison of means 
for grade level and usage was conducted. Teachers who used Thinking Maps® had higher mean 
scores than those who did not. Based on the findings, it may be concluded there appears to be a 
relationship between the perceptions of teachers and consistent usage of Thinking Maps® in 
middle grades science and the increase in achievement score means of the students who used 


them regularly and with fidelity. 
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CHAPTER ONE 

INTRODUCTION 

Middle school students are a distinct group of students with their own characteristics. 
Teaching them can be a challenging and daunting prospect. It often requires multiple ways of 
manipulating information in order for students to form connections to what they are learning. 
One strategy is to use graphic organizers. While utilizing graphic organizers is not a new 
concept, there are always new developments. One of those is Thinking Maps®. 

Purpose of the Study 

The purpose of this study is to identify teacher perceptions concerning the use of 
Thinking Maps® and to examine the effect of implementing Thinking Maps® on middle school 
students’ academic achievement in science. 

Conceptual Framework 

The beliefs that guide the study are generated in part from David P. Ausubel’s work with 
advanced organizers (1960). Ausubel was a psychologist whose Subsumption Theory of 
Meaningful Verbal Learning stated that a very large amount of what enters a student’s brain is 
visual, and students take existing knowledge and enmesh it with new knowledge they receive in 
school (1960, 1963, 1978b). His work with advance organizers, conceptual representations used 
to organize information, literally redefined education, but was not widely known or accepted by 
the critics (1978a). The large body of graphic organizers available today, including Thinking 
Maps®, stem from much of Ausubel’s work. 



Middle school students are a diverse group due to physical, mental, intellectual, and 
emotional growth differences. In one class, a teacher might have students who cannot think past 
the concrete level, students who can think abstractly, and everything in between. This can make 
it incredibly hard for teachers of middle school students to pinpoint techniques and strategies to 
use in the classroom for high percentages of mastery. According to Jean Piaget’s theory of 
cognitive development, middle grades students fall either into the concrete operational stage (7- 
11), characterized by students who are beginning to think abstractly but still may depend on the 
manipulation of concrete objects to help them learn, and the formal operational state (11-15), 
characterized by students moving from concrete-based thinking to greater abstract and logical 
thought, who may not necessarily depend on the use of concrete objects to aid them in learning 
(American Academy of Child and Adolescent Psychiatry, 2012; California Department of 
Education, 1989; Caskey & Anfara, 2014; Edwards, Hopgood, Rosenberg, & Rush, 2000; 
McLeod, 2010). Because of the uncertainty that exists with a student still being in the concrete 
operational stage or in the formal operations stage, teachers require tools and resources that 
allow students in both stages to adequately showcase their work without pointing out the 
difference in the students’ cognitive developmental levels. 

Background 

This study was chosen as a result of the school system in which the researcher is 
employed adopting the Thinking Maps® program at the onset of the 2011-2012 school year. A 
four-person team from each school in the system of eighteen schools was chosen to be on the 
school’s redelivery team. A four-day Training of Trainers event was delivered during the third 
week of June of the summer of 2011, with the intent that each team would redeliver the training 
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to each school’s faculty during the pre-planning period prior to the opening of school. Once the 
initial training was completed by the redelivery teams, each teacher was expected to provide 
samples of students’ work to their principals over the course of the first four months after 
introducing a few of the eight Thinking Maps® at a time. The principals would in turn share 
them at their administrative meetings with the superintendent, the deputy superintendent for 
instruction, and the curriculum directors for the elementary, middle, and secondary levels to 
show the depth at which the program was being utilized. 

Since the researcher was on a redelivery team, she developed an interest in seeing 
evidence of the research that had been presented during the training. Her area of interest was 
middle school curriculum, so her search centered on middle school uses of Thinking Maps® in 
the content area subjects. 

Extensive searches conducted revealed eight quantitative or qualitative studies and many 
scholarly articles in the literature on the effective uses of Thinking Maps® centered on 
elementary schools and their application of the maps in Reading, Writing, Language Arts, and 
Mathematics. Three other studies were completed on secondary and post-secondary education. 
Unfortunately, there was only one study conducted at the middle school level, at Tuttle Middle 
School in Catawba County, North Carolina. The administration reported that from 1994-1996, 
the scores on the North Carolina End-Of-Year Test for writing increased from 43% to 73% for 
the eighth grade (Hyerle, 2000). Hester, Owen, Piekarski, and Hildebran (1995) reported the 
same findings at the same school as an increase in writing, reading, and other achievement test 
scores, but did not include the data from the original report. 



Significance of the Study 

Even though the effects of using Thinking Maps® on student learning outcomes has been 
studied at the high school and elementary levels to a greater extent, there are only a few studies 
focusing on middle level students. This study investigates the extent to which Thinking Maps® 
affect academic achievement in middle grades science. In addition, teachers’ perceptions on the 
use of Thinking Maps® in learning were investigated through surveys. Because of the paucity of 
research at the middle school level, it is important that there is an addition to the literature for 
this developmentally diverse age group, to see if the Thinking Maps® program works and 
improves learning. 

Definition of Critical Terms 

For purposes of this study, the following definitions will apply: 

Academic Achievement is achievement as measured formally by the Georgia 
administration of the Criterion-Referenced Competency Tests (CRCT) and the Georgia 
Milestones Assessments (GMAS). 

Common Core Georgia Performance Standards (CCGPS) are standards adopted for use 
across 47 states, adapted for Georgia, that provide consistency in learning for college, career, and 
workforce development. 

Criterion-Referenced Competency Test (CRCT) is a non-standardized test developed and 
administered in the state of Georgia to measure state standards instruction in grades 2-8. 

Georgia Milestones Assessment System (GMAS) is the standardized test developed and 
administered in the state of Georgia to measure state standards instruction in grades 3-12. 
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Georgia Performance Standards (GPS) are instructional and curriculum standards 
developed by the Georgia Department of Education with specific standards at each grade level, 
tested by the Georgia CRCT. 

Graphic organizer is a tool that organizes information visually. 

Middle school is a school incorporating grades 6 through 8. 

Thinking Maps® is a set of eight visual-verbal organizers developed by David N. Hyerle 
(1993) that uses specific thinking processes in which to organize data, ideas, or information: 

Brace Map is one of eight Thinking Maps® that identifies whole-to-part 
relationships. 

Bridge Map is one of eight Thinking Maps® that shows relationships in 
analogies. 

Bubble Map is one of eight Thinking Maps® that describes a topic. 

Circle Map is one of eight Thinking Maps® that defines a topic in context. 
Double Bubble Map is one of eight Thinking Maps® that compares and contrasts 
two topics. 

Flow Map is one of eight Thinking Maps® that sequences an event or gives 
directions or instructions. 

Multi-Flow Map is one of eight Thinking Maps® that shows cause-and-effect 
relationships. 

Tree Map is one of eight Thinking Maps® that categorizes, classifies, or 


organizes information. 



Assumptions of the Study 

The assumptions within this study are: 

1. Teachers use Thinking Maps® as a result of a system-wide instructional initiative. 

2. Students use Thinking Maps® in all content-area and connections classes. 

In addition, all students take the Georgia Criterion-Referenced Competency Test in April, 
which includes grades 6-8 at the middle school level. During the 2014-2015 academic year, 
middle schools will administer the Georgia Milestones Assessments (GMAS) in place of the 
Georgia CRCT and Georgia EOCT. Results of this study are applicable only to the Georgia 
school system in which it takes place. 

Limitations of the Study 

The limitations of the study include: 

1. Individual differences in teachers. 

2. Teachers have varying levels of teaching effectiveness due to different levels of expertise 
and professional training. 

3. Students in each grade level’s content-area classes may not receive the same instruction 
using Thinking Maps®. 

4. Teachers new to the school system may not have the same experience or staff 
development in the implementation of Thinking Maps® as those teachers who received 
instruction in the use of Thinking Maps® during the initial and subsequent training. 

5. Students may have different learning styles or preferences, and academic achievement 


levels. 
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Delimitations 

1. The population of this study is delimited to middle grades students in grades 6-8. 

2. This study is delimited to science classes. 

3. The population of this study is delimited to students in a mid-sized school system in 
west central Georgia. 

Biases of the Study 

A bias applicable to this study could be that if a teacher was a member of the redelivery 
team, then he/she may have a greater buy-in or vested interest in the usage of Thinking Maps® 
in his/her classroom. Additionally, the teacher might have a deeper knowledge of the inner 
workings of Thinking Maps® and its use. Conversely, a teacher who was not a member of the 
school’s redelivery team may refrain from proper utilization of Thinking Maps® within the 
middle school curriculum or not use them at all. 

Personal Statement 

The researcher was chosen to be on the Thinking Maps® training of trainers team from in 
2011. The team included a teacher from each content area: English/language arts, math, science, 
and social studies. The assistant superintendent for instruction had seen a presentation on 
Thinking Maps® and selected it as the newest school improvement initiative, so the objective 
was to learn about and redeliver the training to colleagues. 

As an educator, the researcher was initially intrigued in the Thinking Maps® program 
because of the potential use in the classroom as a way for students to organize and manipulate 
information. She had also been on the training team for Max Thompson’s Learning-Focused 



Schools initiative, which included many graphic organizers whose functions were the same as 
some of the Thinking Maps®. Because she believed students must have multiple encounters 
with information to make permanent connections to learn content, she valued the use of graphic 
organizers, which included some Thinking Maps®. However, there were also other ways in 
which students evaluated and used data or information that may supersede Thinking Maps® 

(e.g., Venn diagrams, T-charts, outlines). Due to the diverse training the researcher had involving 
the use of graphic organizers, she believed she could draw objective conclusions regarding the 
use of graphic organizers and Thinking Maps®, and personal bias will be reduced more as a 
result than if she had only training in Thinking Maps® and no other formal training in graphic 
organizers to which she could make a comparison. 

Statement of the Problem 

Educators constantly seek new instructional tools or innovative programs when student 
learning outcomes are not at the desired level. When so much emphasis is on standardized 
testing, there is concern that students will have limited opportunities to engage in critical 
thinking and higher-order skills such as developing analogies and identifying cause-and-effect 
relationships. The challenge, then, is maintaining high standards while teachers teach every 
child and leaving no one behind. 

To meet this demand, it is crucial to provide teachers with quality tools and resources to 
help students succeed in the classroom. Teaching middle grades students can be particularly 
challenging, as current brain research indicates that the brain is not ready for true abstract 
thinking until the early to mid-twenties. Giving students the tools with which to organize 
information is one strategy educators can utilize to develop abstract and critical thinking skills, 



9 


aid in concept acquisition and retention, and make connections in the content areas. One 
possible means to give students the tools to organize information is to use the Thinking Maps® 
program to connect new knowledge and think critically. Thinking Maps® are a series of eight 
visual information organizers developed for eight basic cognitive processes: defining, 
describing, comparing and contrasting, classifying, sequencing, causes and effects, whole-to-part 
relationships, and analogies. But, do Thinking Maps® help students with content attainment? 
This study is concerned with the educational progress of middle grades students using Thinking 
Maps®, and therefore, it is important to study Thinking Maps® as a viable resource to help 
students learn science in the middle school. 

The brace map Thinking Map® was designed to break information, such as the systems 
of the body, the parts that make up the electromagnetic spectrum, or the types of rocks into the 
principal parts in the whole-to-part relationship. Doing so allows students to see visually how 
objects or ideas can be broken down into smaller parts. In this way, students can better see how 
the smaller parts may work together to make up the whole, building a spatial relationship as well 
as an organizational chart. 

The bridge map Thinking Map® was designed to help students create analogies. 

Creating analogies can help students think more deeply, increasing their critical thinking and 
reasoning abilities and helping them to build their ability to think abstractly. By increasing the 
ability to reason and think critically students’ ability to solve problems and reason better as to 
why something occurs. For example, to introduce students to the types of chemical reactions, the 
analogy of couples at a middle school dance and the different combinations of partners that are 
created helps students to understand better what happens. With a synthesis reaction, where 
carbon and oxygen react to generate carbon monoxide, an analogy might be to have a boy plus a 



girl equal a couple on the dance floor. In a double-replacement reaction, if hydrochloric acid 
(HC1) and sodium hydroxide (NaOH) are combined, the two dancing “couples” break apart and 
change “partners,” generating sodium chloride (NaCl) and water (FEO). Thus, the students are 
able to visualize the changes during each reaction type by visualizing the different partner 
combinations, leading to a better understanding of chemical reactions. 

The bubble map Thinking Map® was designed to describe characteristics, properties, or 
qualities of the attributes of something. In science, the bubble map could be used to describe the 
characteristics of a specific layer of the atmosphere or the ocean, the properties of solids, liquids, 
gases or plasmas, or describe the characteristics of a particular biome, such as the taiga or 
maritime forest. Being able to describe characteristics of a specific scientific concept leads to a 
better understanding of how that concept contributes to the content being taught. For example, if 
students know the properties of the four main states of matter, they are better able to understand 
the molecular motion of each state and how the four states function in the natural world. 

The circle map Thinking Map® was designed for students to generate lists of what they 
know about a topic, brainstorm, or relate information associated with the prior knowledge they 
might have of an impending topic. In the science classroom, this might be helpful used as a 
KWL or pre-assessment to gauge the prior knowledge students have about content. 

Additionally, it could be used to brainstorm on a topic. This could including anything from what 
students know about the clastic sedimentary rock group, animals existing in the maritime rain 
forest, to information about the Periodic Table of the Elements. 

The double bubble map Thinking Map® was developed as being similar to the Venn 
diagram, where two things are compared and contrasted. This map is pertinent to the science 
classroom in that many comparisons are made within a topic being taught. For example, 
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students might compare and contrast shield and cinder cone volcanoes, animal and plant cells, or 
homogeneous and heterogeneous mixtures. Students using comparative thinking improve their 
critical thinking, which increases their comprehension and leads to better learning of the concept. 
Students comparing shield and cinder cone volcanoes would have to remember that the lava 
emitted from a shield volcano flows faster and over a larger area than that of a cinder cone, 
which would help the student to realize that because of this, the lava from the shield volcano has 
a lower viscosity. 

The flow map Thinking Map® started as a response to flow charts, showing the 
sequences of events, timelines, directions, procedures, steps, or the order in which things occur. 
All of these are especially relative to the science classroom. Students are able to make greater 
connections to the content they are learning if they know processes in their correct order. For 
example, students tracking the progression of the history of atomic theory trace how the model of 
the atom developed over time and can better see the advances made in atomic theory as a result. 
Another example is recounting or retelling the steps of how to complete the procedures of a lab 
activity, which spans all grade levels. Being able to recall the steps leads to better understanding 
of the process of the activity, whatever it happens to be. Greater comprehension occurs as a 
result, just as it does when having a student retell a story she read. 

The multi-flow Thinking Map® was designed to show cause-and-effect relationships, or 
to show the consequences of an event. In the science classroom, it gives students the opportunity 
to predict what they think will happen during an investigation and show the results. For 
example, students investigating acids and bases predict the identity of common substances and 
proceed to test the substances using an acid-base indicator. The color the indicator turns the 
substance being tested shows the presence of an acid or a base, and the student can then record 



the effect or the result. Students can take the information gleaned from the investigation and 
produce an argument based on their results. Being able to argue a position effectively leads to 
increased abilities in critical thinking and reasoning, as well as in scientific communication. 

The tree map Thinking Map® was designed for students to classify, sort, group, 
categorize, and organize information, which is invaluable in the science classroom. Students can 
use the tree map to organize information, which then is used as an aid to study. For example, 
students can generate a tree map on ocean zones and show pertinent facts and information they 
research or write down as they learn about them. Another example is generating a tree map for 
conduction, radiation, and convection during the study of thermal energy. The students can then 
add examples, illustrations, or diagrams to help them commit the information to short- and long¬ 
term memory. 

Using a program such as Thinking Maps® may help students process and retain 
information in their science classroom. The maps provide visual-spatial aids on which students 
can record information they learned or discovered. With so many different types of graphic 
organizers and visual-spatial resources available in education today, the Thinking Maps® 
program may narrow the field for middle school students and aid them in learning science 
content and concepts much easier than deciding ways to record what they need to know. 

Research Questions 

The following research question will guide this study: 

1. To what extent do Thinking Maps® affect middle grades students’ academic 


achievement in science? 
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2. What are teachers’ perceptions about the use of Thinking Maps® with middle grades 
students? 

3. What, if any, is the relationship between the use of Thinking Maps® and teachers’ 
perception and implementation in middle grades? 

Summary 

Educators are always looking for strategies, resources, and programs to help their 
students learn. One program being used is Thinking Maps®, which was developed by David 
Hyerle to take the place of the vast number of graphic organizers available and use eight basic 
brain processes he identified to organize and interact with information so that students can think 
critically and leam the concepts being taught. This study examines the use of Thinking Maps® 
in content-area learning to see if higher usage yields better results in content acquisition. 

Teachers will be surveyed for their thoughts and perceptions of the program and if they believe it 
does what it was intended to do: help students learn science content. This study is significant in 
that if the program does not contribute to learning, another avenue for students to use to learn 
science content should be investigated. 



CHAPTER TWO 


REVIEW OF LITERATURE 

With closer scrutiny and ever-increasing demands on education from the U.S. 
government for many years in the guise of the Elementary and Secondary Education Act and the 
No Child Left Behind Act of 2001, and more recently A Blueprint for Reform: The 
Re authorization of the Elementary and Secondary Education Act (2015), along with other 
mandates, states and local school systems have struggled to increase student successes. 

Educators are looking closer at research-based instructional strategies and educational practices 
to get the most out of their carefully measured budget, with their students’ academic 
achievement centrally in focus. Additionally, new theories about how the teen brain learns have 
been recently hypothesized giving credence to some of the theories, strategies and practices used 
by the nation’s educators, including brain-based learning, metacognition, graphic organizers, and 
another incarnation of visual organizers, the Thinking Maps® program. This study will focus on 
the use of Thinking Maps® in middle grades science to determine if there is a statistically 
significant effect on student achievement. Teachers’ perceptions of the use of Thinking Maps® 
will also be examined to determine if they believe there is a positive impact on student learning 
and to investigate the relationship between these perceptions and student achievement. 
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Brain-Based Learning 

Eric Jensen (2008, p.4) defined brain-based learning in A Fresh Look at Brain-Based 
Education as the “engagement of strategies based on principles derived from an understanding of 
the brain,” and McCall (2012, p. 42) defined brain-based learning as “an approach that reflects 
the interrelatedness of the mind, brain, and body.” While there should be no doubt that all 
learning has something to do with the brain, the issue at hand right now is what strategies 
educators should utilize to maximize their students’ learning. Our current limited knowledge of 
exactly how the brain works is keeping this issue in the forefront of educational debate. What we 
currently know about how the brain learns, that is, the science of the brain, and the educational 
practices used by educators need to be cohesive, meaning educators need the tools and resources 
that best help the student learn according to what we know about the brain. Educational practices 
should not include the fad strategy of the moment, even though meta-analyses indicate brain- 
based learning “leads to greater academic achievement than traditional teaching methods” 
(Edelenbosch, Kupper, Krabbendam & Broerse, 2015; Goziiycsil & Dikici, 2014, p.646). 

Because we do not know all there is to know about how learning occurs, we should be very 
careful when using any teaching strategies that might have been developed from the results of 
neuroimaging studies (Willis, 2007). A study by Pashler, McDaniel, Rohrer, and Bjork (2008) 
concluded the widespread use of learning-styles measures in educational settings is unwise and a 
wasteful use of limited resources. An example of this would be teaching students based on what 
has been identified as “visual learning style.” 

Jensen (1996 & 2005) stated that visual images make up 90% of the information that 
enters the brain, and the eyes register 36,000 visual images in an hour. The brain devotes the 
greatest part of its information processing to these visual images as compared to non-visual, 
abstract information (Plotnik & Kouyoumdjian, 2014). Even with that, the brain does receive 



information from all five senses, leading researchers to believe there are really no specific 
learning styles, but what matters the most is being exposed to content in multiple ways (Pashler, 
et al., 2008). Other researchers have shown that emotion and personal connectivity contribute to 
a person’s ability to pay attention in order to learn information necessary to the topic (Yeager, 
2004). In the PBS series The Secret Life of the Brain, the statement “the brain is not a thinking 
machine, but a feeling machine that thinks” (Grubin, 2001) shows what the ancient Egyptians 
knew to be partially true: emotion, even if it doesn’t come directly from the heart, is important 
to learning. The brain takes the influx of information coming into it from different sources, 
discards what it determines to be irrelevant, then makes connections within the neural network, 
and places the necessary information into long-term memory for later use and retrieval (Smith & 
Kosslyn, 2013). Making visual patterns, such as flow charts, analogies, or taxonomic charts to 
make sense of the huge amounts of information coming in is a task in which the brain excels. 
Yeager (2004) said that when students can link abstract ideas to a visual image, the patterns can 
independently alert the brain to search out a recognizable organization in which to store the new 
information. When the students become familiar with these organizational patterns, they develop 
the ability to think and learn at higher levels, which in turn increases the amount of activity in the 
brain and can lead to an increase in deeper thinking and retention. Some of the newest studies 
have identified three brain systems that may process visual images, one each for shape, color, 
and motion or perspective (Society for Neuroscience, 2012). Ultimately, areas of brain activation 
are able to use the graphic representations of these basic brain functions and patterns for 
increased ability to think in the upper levels of Bloom’s Taxonomy (Brown, 2003). 

In any event, there is a consensus among those in neuroscience and education that caution 
must be observed when taking information from brain-based studies and applying it to education 
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(Willis, 2007). It is accepted that for students to learn, new content must be meaningful to the 
learner’s experience or it will be disregarded by the brain as unnecessary (Bellah, Robinson, 
Kaufman, Akers, Haase-Wittier, & Martindale, 2008). However, until a clear definition of how 
students’ brains place importance (metacognition) on the knowledge, organize it, and store it has 
been determined, the changes to the foundations of education will not occur (G6ziiye§il & Dikici, 
2014). Willis (2007) may have said it the most succinctly: 

For now, a combination of the art of teaching and the science of how the brain responds 
metabolically to stimuli will be the best guide for educators in their efforts to find the best 
neuro -logical ways to present information in ways that potentiate learning, (p. 699) 

Metacognition 

Metacognition has been defined in education as how people think about thinking, or their 
awareness of what they know or do not know (Flavell, 1976). It can also mean those abilities we 
have and use to get meaning from something we have read or the steps we take to solve some 
kind of problem (Flavell, 1976). While knowledge of the process of thinking about how one 
thinks has been around for many years, it is most commonly associated with developmental 
psychologist John H. Flavell, a professor at Stanford University, who first used the term 
“metacognition” in 1976. According to Flavell, there are two distinctive divisions of 
metacognition: metacognitive knowledge and metacognitive regulation (1976, 1979). 

Metacognitive knowledge focuses on what one knows about the cognitive processes and 
how to control or accommodate for them. Metacognitive knowledge can be subdivided into 
three categories: person variables, task variables, and strategy variables (Teaching Excellence in 
Adult Literacy (TEAL) Center, 2012). Person variables refer to being aware of one’s own 



strengths and weaknesses in being able to learn new information or a new skill, task variables 
involve being able to decide what is known about a task, purpose, or what is needed to complete 
a task, and strategy variables are those used to activate the prior knowledge about a task so that a 
plan of attack to accomplish the task can be formulated (TEAL, 2012). A singular example 
given by Livingston (1997) streamlines Flavell’s idea, “I know that I (person variable) have 
difficulty with word problems (task variable), so I will answer the computational problems first 
and save the word problems for last (strategy variable) (Section 4, paragraph 5).” This ability to 
look at a problem or task, surmise the deficiencies in knowledge of the topic, and generate a plan 
to increase the knowledge and develop a plan to execute the task affords students greater 
problem-solving and critical thinking skills to be successful at all levels (Joseph, 2010). 

Metacognitive regulation refers to how a person regulates or controls his own learning by 
using three critical skills: planning, monitoring, and evaluating (Efklides, 2014; Livingston, 
1997). The planning phase requires thinking about what is known about a task topic, choosing 
an appropriate plan of attack and acquiring the appropriate information needed to carry out the 
task (Livingston, 1997). This planning phase might include selecting an organizer to use to 
organize information in order to manipulate it or use it to solve the task or problem. When 
monitoring progress, performance must be taken into account to determine if the goal is being 
met. Evaluating the completed task entails the appraisal of the steps taken to complete the task 
as well as the quality of the finished product. Examples of this could be the planning and 
execution of a paper on the similarities between Gandalf in The Lord of the Rings and Jesus 
Christ, or designing and building a Rube Goldberg machine to drop pet food into a bowl under 
certain conditions. The success or failure of the product is not only due to the finished product, 
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but the steps the student had to take, sometimes implicitly, in order to complete the task to show 
what he learned. 

It may be that students do not have the innate metacognitive ability to realize what they 
know or don’t know, or plan, monitor, and evaluate their products, but these skills developed 
(Joseph, 2010; TEAL, 2012; Turner, 2011). Some strategies for developing metacognition 
include modeling the thinking-aloud process, activating prior knowledge, outlining or other 
forms of note-taking, pre-writing strategies, using graphic organizers such as anticipation/ 
reaction guides or concept maps, generating mnemonics, breaking down words into roots, 
prefixes, and suffixes, using checklists, and utilizing problem-solving strategies, such as 
deconstructing questions or problems, to name a few (Efklides, 2014; Joseph, 2010; Livingston, 
1997; Pacheco & Goodwin, 2013; Ritchhart, Turner, & Hadar, 2014; TEAL, 2012). Since 
Thinking Maps® allow students to organize and represent information in different ways, it can 
be hypothesized that they support students’ metacognitive thinking based on how literature 
identifies metacognitive thinking strategies as described above. 

Adolescent Learners 

Middle school students are a diverse group given their developmental and cognitive 
differences. In one class, a teacher might have students who cannot think past the concrete level, 
students who can think abstractly, and everything in between. This can make it incredibly hard 
for teachers of middle school students to pinpoint techniques and strategies to use in the 
classroom for high percentages of mastery. 

Granville Stanley Hall, the first president of the American Psychological Association, 
was one of the first psychologists to identify adolescents as a disparate group in his book, 
Adolescence: Its Psychology and Its Relations to Physiology, Anthropology, Sociology, Sex, 



Crime and Religion, in 1904 (Caskey & Anfara, 2014). Later, Jean Piaget’s work in the stages 
of cognitive development set the groundwork for change in education. He determined that 
students around the ages 7-11 were in the concrete operational stage. The concrete operational 
stage is characterized by students who are beginning to think abstractly but may still depend on 
the manipulation of concrete objects within the learning process (Piaget, 1969). In the earlier end 
of this age range, students may evaluate logical thinking with only concrete evidence (Edwards, 
et al., 2000). Their thinking changes over time with the significant changes in the brain’s 
prefrontal cortex, leading to increased ability to conserve, plan, classify, make decisions based 
on reasoning, and determine possible consequences, with an increased attention span (Caskey & 
Anfara, 2014). Subsequently, learners at this level need more structured learning experiences and 
activities (Caskey & Anfara, 2014). The structured learning experiences help students bridge the 
gap between concrete and abstract. 

Following concrete operations, the next stage of Piaget’s theory of cognitive development 
is the formal operational stage, which encompasses the 11-15 year age range (Piaget, 1969). It is 
characterized by students moving from concrete-based thinking to greater abstract, logical 
thought that does not necessarily depend on the use of concrete objects in the learning process 
(American Academy of Child and Adolescent Psychiatry, 2012; California Department of 
Education, 1989; Caskey & Anfara, 2014; Edwards, et al., 2000; McLeod, 2010). The two main 
characteristics of learners at this level include the ability to deductively reason hypotheses and 
possible solutions to solve a problem, and a propositional focus in which they can logically 
evaluate the validity of assertions without referencing real-world situations (Edwards, et al., 
2000). Further, students at this stage develop greater ability for independent thought and 
metacognition, that is, knowing and being aware of what they know and do not know (California 
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Department of Education, 1989; Caskey & Anfara, 2014). Abstract thinkers benefit from 
challenging, less structured learning experiences (Caskey & Anfara, 2014). 

Given the developmental and cognitive challenges of middle grades students, it is 
necessary for teachers to be aware of practices and resources to help their students learn. 
Providing learning opportunities for students in both concrete operational and formal operational 
stages makes sure students can learn and grow into abstract thinkers. 

Graphic Organizers 

While the term “graphic organizers” did not appear until the last half of the 20 th century, 
these visual tools have been used for hundreds of years to organize information for various 
purposes: church and religious services, merchandise inventories, will asset lists, timelines, 
comparisons between sets in mathematics, charts, matrices, and many others. David Ausubel’s 
(1960) ground-breaking work evolved into the theory that students could learn information 
quicker and with better retention if they had a structured visual for organizing the information. 
This led to the development of many of the graphic organizers that we see today (Gibbs, 2009). 
In addition, the work of some of the best education minds of the day, such as Robert Marzano, 
Carol Ann Tomlinson, Jay McTighe, Howard Gardner, Thomas Armstrong, Grant Wiggins, and 
Dinah Zike, among many others, endorsed the use of graphic organizers in differing formats as a 
way for students to condense large amounts of information, organize it, and scaffold thinking 
around it (Gardner, 1983; Marzano, 2016; Tomlinson, 2010; Wiggins & McTighe, 2005; 
Armstrong, 1993). Furthermore, using these visual graphic representations for taking organizing 
concepts has been immensely helpful for students to take what they have read and make sense of 
it by constructing their own graphic representations in the Thinking Maps’® formats because 



they can add their own creativity (Brown, 2003). Students are able to plan what they will do and 
follow through, making the information they find more easily understood (McKenzie, 1997). 
These graphic images give students something concrete that can aid them in making connections 
between what they have read and their own understanding of it (Coleman, 2003). Graphic 
organizers may also help students with disabilities make sense of the loads of information given 
to them (DiCecco & Gleason, 2002). DiCecco and Gleason (2002) conducted a study of middle 
school students with reading disabilities and found that students had significantly greater 
knowledge of the subject after using graphic organizers than those students who did not. 
Halvorson (2010), Dexter (2010), and Hernandez (2014) found that Title I students and students 
with disabilities did not like using graphic organizers but felt using them helped the students 
learn new content vocabulary with the addition of a visual aid markedly improving the learning 
of content-area curriculum, and increasing their attitudes toward reading and participating. In a 
study completed on students with disabilities in mathematics, Cully (2010) found they made 
better progress than their non-disabled peers when using graphic organizers. Manoli and 
Papadopoulou (2012) concluded that graphic organizers were “conducive on [sic] assisting 
students in activating prior knowledge, gaining insight into text structure, identifying as well as 
connecting the main ideas of a text resulting in better recall and retention of information” (p. 
354). Cleveland (2014) found that while no clear answer could be made for or against the use of 
graphic organizers as an aid for organizing information or studying, the data indicated that when 
a student’s cognitive load was very high, meaning the student is dealing with many concepts at 
once, “graphic organizers acted as a tool to facilitate learning” (p. 145). Chemistry students’ 
attitudes for both chemistry and for the use of graphic organizers to help them learn key 
chemistry content grew more positive (Torres, Espana, & Orleans, 2014). McKinney (2007) 
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reported “the higher mean score on the chapter tests by the experimental groups in comparison to 
the control groups show promise in using graphic organizers in the [secondary] science 
classroom” (p. 77). When engaged in activities that required students to use critical thinking, 
graphic organizers were helpful (Torres, Espana, & Orleans, 2014). Unfortunately, there are so 
many available ways to organize data and information that it can seem daunting to the student 
who attempts it. Being able to pare down the sheer volume of graphic organizers available into a 
more manageable number for students to remember may help students make better use of these 
tools. This is where a program like Thinking Maps® could make a difference. 

Thinking Maps® 

During his doctoral research at the University of California at Berkeley in the early 
1990’s, David Hyerle focused on graphic organizers and how they could help students. He put 
his efforts toward a system to “flexibly pattern information in order to construct understandings” 
(Hyerle, 2004, p. 12), which could lead to higher levels of thinking in the analysis, synthesis, and 
evaluation ends of Bloom’s Taxonomy (Gibbs, 2009). His research showed that “early mind¬ 
mapping techniques (in the 1970’s) facilitated open-minded thinking yet lacked the consistent 
structure and deeper levels of complexity required for today’s classrooms” (Hyerle, 2004, p. 7). 
Using the brain research of the time, Hyerle identified eight fundamental thinking processes that 
the brain uses to store and use information: defining, describing, comparing and contrasting, 
classifying, sequencing, cause and effect relationships, whole-to-part relationships, and 
analogous relationships (Hyerle, 1993). Using the cognitive processes and the graphic 
organizers that educators were utilizing at the time, Hyerle developed his own primitive graphic 
organizers, and by using the “maps” with the intent that students could relieve some of the load 



on their short-term memory which would allow increased mental capacity to think at deeper 
levels with more understanding of the concepts they were learning, be more creative, and think 
more abstractly than was formerly possible (Brown, 2003). These graphic organizers evolved 
into what are now known as Thinking Maps®, eight specific maps to visually organize 
information. Because Hyerle developed each graphic representation to follow the pattern of the 
brain process for which he claims it stood, the natural tendencies of the brain could now be 
utilized to their fullest potential (Hyerle, 1993). For example, a student could use a tree map to 
organize facts about conduction and convection that could then be used in a double-bubble map 
to make comparisons. Hyerle (2004) stated, 

The combined practical, theoretical, and critical attributes of these different types of 
visual tools have informed the continuing evolution of Thinking Maps® into a 21 st 
century language for learning, synthesizing many of the best qualities of these other types 
of visual tools: an evolution from the generative quality of brainstorming webs, the 
organizing structure of graphic organizers, and the deep cognitive processing found in 
concept maps (p. 7). 

This flexibility enables students to take those basic visuals, which are systems of 
interconnected information, and organize their thinking by taking the content knowledge, 
transferring it to their visuals, and adapting it for themselves (Gibbs, 2009). Students can add 
illustrations or diagrams, color, symbols, mnemonics, and other embellishments to help them 
learn and remember content. This process ensures that the focus is on the affective domain, 
where students can develop metacognitive skills (Hubble, 2004). 

The eight maps Hyerle developed are: the Circle Map, the Bubble Map, the Double 
Bubble Map, the Tree Map, the Flow Map, the Multi-Flow Map, the Brace Map, and the Bridge 
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Map. Collectively, they are called Thinking Maps®. According to Hyerle, each map 
corresponds to a fundamental cognitive process that allows people to compartmentalize 
information. For example, the double-bubble map is used for comparing and contrasting, which 
leads the student to find similarities and differences between the two items being compared. The 
tree map is used to organize facts under categories, and facts are chunked to be easily found or 
recognized. The flow map is used to show the progression of ideas or the procedures to 
accomplish a task, and the students must be able to distinguish the order in which an event 
occurs. The bridge map is used to build analogies, where students make comparisons between 
objects based on similarity or comparability with similes and metaphors. The brace map breaks 
something into its component parts, forcing students to think about the parts of a whole and how 
those parts fit together. The multi-flow map can show cause and effect relationships, making the 
student focus on the why and how something happens. The circle map is used to define a topic or 
to show the characteristics that make the topic what it is, such as the characteristics or properties 
of a liquid. Lastly, the bubble map can describe a topic, using adjectives to give a student the 
opportunity to use language to describe what he knows about a subject. When compared to the 
myriad of graphic organizers available, the Thinking Maps® are relatively static, which allows 
for use across many content areas, not only in education, but in business, medical professions, 
the military, and other applications as well. Hyerle (2004) believed there were five qualities to 
his Thinking Maps®: consistency, flexibility, developmental appropriateness, favoring 
integration, and reflection, and they could be utilized on a global level to distribute information. 
In addition, maps used for any class, be it math, English, foreign language, or wood shop use the 
same design, and the repetition and consistency in organizing information makes it helpful for 
students to organize information efficiently and use in across all content areas to enhance their 



thinking (MacIntyre, 2004). These organizational patterns that can be utilized across disciplines 
included identifying similarities and differences, summarizing and note-taking, non-linguistic 
representations, and advance organizers (Dean, Stone, Hubbell, & Pitler, 2012; Marzano, 
Pickering, & Pollock, 2001). This universal use is important as Thinking Maps® collectively 
can be used as advance organizers and non-linguistic representations. Furthermore, similarities 
and differences can be represented in a Double-Bubble Map and a Tree Map can be utilized for 
summarizing and note-taking, for example, where students use the organizers to show what they 
know. 

After twenty years of implementation, there is evidence for the usage of Thinking 
Maps®, most of which involved direct support and training from its creator. David Hyerle 
(2000) cites the results of several schools’ implementation of Thinking Maps® in A Field Guide 
to Using Visual Tools. Margaret Fain Elementary School in Atlanta, Georgia, on the Georgia 
Test of Basic Skills, the forerunner of the Georgia Criterion Referenced Competency Test 
(CRCT), reading scores increased from 29% to 69%, and math scores rose from 32% to 63% in 

1996. At Friendship Valley Elementary School in Carroll County, Maryland, on the Maryland 
School Performance Assessment Program (MSPAP), writing scores increased 27%, language 
scores rose 20%, and science scores rose 18.2% in 1996 after implementation. Additionally, 
Windemere Elementary School in West Orange County, Florida, on the Stanford 8 Achievement 
Test, reading scores increased from 68% to 80%, and math scores rose from 79% to 92% in 

1997, and Waitz Elementary School, a 100% Title I school in Mission County, Texas, on the 
Texas Assessment of Academic Skills (TAAS), math scores increased from 41.2% to 76.5%, 
while reading increased from 62% to 88% in 1994 following initiation of the program. At 
Claremont Elementary School in Catawba County, North Carolina, scores on the fourth grade 
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North Carolina End-of-Year Test in writing showed an increase from 33% to 68% over the years 
1993-1996, while at Chadbourne Elementary School in Columbus County, North Carolina, 
writing scores on the same test increased from 35% to 60% from 1993-1995, and at Morrisville 
Elementary School in Wake County, North Carolina, the writing scores increased from 32% to 
76% from 1993-1996. Lastly, A. T. Allen Elementary School in Cabarrus County, North 
Carolina, on the North Carolina State End of Year Tests in 1998, math scores rose from 80% to 
91%, reading scores increased from 77% to 89%, and writing scores rose from 29% to 77%. The 
one middle school mentioned by Hyerle was Tuttle Middle School in Catawba County, North 
Carolina, showed an increase in eighth grade writing scores on the North Carolina End-of-Year 
Tests from 43% to 73% from 1994-1996 (Hyerle, 2000). While these are impressive statistics, 
these were the reported significant gains; all scores were not mentioned and could tend to make 
the more positive scores suspect. 

Most of the studies focused on elementary-age students. Gibbs (2009) tested to see if the 
reading achievement of third and fourth grade students would improve if they implemented the 
use of Thinking Maps®. Using the Tennessee Comprehensive Assessment Program (TCAP), the 
scores were analyzed in independent sample Mcsts. Results indicated there was no evidence the 
length of implementation affected reading achievement, but there was a significant increase in 
student reading scores (Gibbs, 2009). Another study completed by Russell explored the impact 
of Thinking Maps® on reading comprehension achievement on fourth and fifth grade students 
using a repeated measures ANOVA and perception surveys. No significant difference in the 
reading achievement on the Texas Growth Index (TGI) for socio-economic status, ELP, gender, 
or ethnicity was noted, but there was a significant difference between the fourth and fifth grade 
scores (Russell, 2010). Sunseri studied the effects of using Thinking Maps® to generate 



expository writing. She worked with fourth graders, ELL and non-ELL students, and found no 
significant difference between the treatment and control groups. However, there was a small 
difference in the ELL students’ writings as compared to the non-ELL students’ writings (Sunseri, 
2011). Leary (1999) studied the effects of Thinking Maps® on the achievement scores of fourth 
graders in math, reading, and language arts. He used a 3-way ANOVA analysis of Stanford-9 test 
scores. Results indicated no significant difference between the treatment group and the control 
group on any of the variables, including race, gender, and previous achievement level (Leary, 
1999). Also working with fourth grade students, Brown explored the students’ use of Thinking 
Maps® to develop high-level reading comprehension tests. She found, using paired samples t- 
tests, that students showed significant improvement in both their ability to generate higher-order 
questions, and to answer questions generated by their peers (Brown, 2003). Hickie (2006) tested 
to see if an association existed between Thinking Maps® usage and student achievement in 
reading, English/language arts, and math. She studied fifth grade students and used paired 
sample t-tests and 3-factor repeated measures ANOVA to analyze the data. The results showed 
significant differences between the means for reading from 2003 and 2005, but no significant 
difference in the math means. There was also a significant difference between the English/ 
language arts mean scores for 2003 and 2005 (Hickie, 2006). 

Robert Slavin and his colleagues (2009) at Johns Hopkins University conducted a meta¬ 
analysis of 79 upper elementary reading studies and their effect on reading achievement in 2009. 
They looked at two separate studies on Thinking Maps® in grades four and five by Leary (1999) 
and Hickie (2006), and noted the program has promise in helping students make gains in their 


reading achievement. 
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A study completed in 2013 on the effects of Thinking Maps® on the science achievement 
of 170 fifth graders in a northeast Georgia school system revealed the treatment group using the 
Thinking Maps® program underscored the non-treatment group by a mean of 23.8 points on the 
Georgia Criterion-Referenced Competency Test (Hudson, 2013). 

According to Ball (1998), Thinking Maps® aided both traditional and nontraditional 
college students to increase their ability to retain what they read. In this study, the treatment 
group using Thinking Maps® outperformed the control group, in which the instruction was the 
same with no graphic organizers were endorsed, in 5 of 7 variables. The treatment group 
outperformed the control group in fast reading, comprehension, structure, vocabulary, and word 
parts. With each of these, p < .01. The other two variables were phonics and scanning. This 
could translate to middle school students by aiding them in content knowledge acquisition by 
acting as a study aid. Another study on college-age students was completed by Gallagher 
(2011). Students in the Advanced Nutrition and Metabolism classes at East Carolina University 
improved the organization and the clarity of their writing to evaluate dietary literature over four 
semesters. Students interviewed indicated they felt better about their ability to organize and 
write clearly and they were able to do it in a more concise manner. 

Unfortunately, there is little data available for the middle grades regarding the use of 
Thinking Maps®. As reported by Hyerle (2000), Tuttle Middle School in Catawba County, 
North Carolina had an increase in scores on the North Carolina End-Of-Course Test in writing. 
From 1994 to 1996, students showed an increase at the eighth grade level from 43% to 73% 
(Hyerle, 2000). In a related study, DeCecco and Gleason (2002) examined if graphic organizers 
helped middle school students with learning disabilities. They showed that while the students 



scored significantly higher on their post-tests, they also showed greater knowledge of the topics 
they studied than did the students who did not use graphic organizers. 

Anecdotal records from over thirty North Carolina educators using Thinking Maps® 
indicate that many teachers are in favor of the usage of Thinking Maps®, some because 
Thinking Maps® help students navigate through information to which they are exposed, but 
because they help support thinking and help students explain and understand what they are 
learning (MacIntyre, 2004). Further, there is evidence that students who utilize information in 
different forms using Thinking Maps® in conjunction with other practices where students 
manipulate content-area knowledge in different ways, known as double-and triple-processing, 
gain a much deeper understanding of the content they are learning (Hubble, 2004). For example, 
during an initial lecture, a science teacher might have students outline the information presented 
on the history of atomic theory, which is known as structured note-taking (Thompson, 2003), 
then taking the information and developing a Tree Map to categorize specific information for 
each scientist involved, then developing a Flow Map showing the sequence of events in the 
development of atomic theory. Being immersed in the information and material in different 
formats enables students to better grasp what they need to know. 

Summary 

The review of the literature suggests the effectiveness of the use of graphic organizers 
and Thinking Maps® is somewhat mixed in its support of student learning in different content 
areas, namely reading, language arts, and mathematics at the elementary level. However, 
evidence has shown its use at the secondary and collegiate level has assisted students in their 
learning. Ball (1998) found Thinking Maps® helped both traditional and nontraditional college 
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students to increase their ability to remember and retain what they read. Additionally, Gallagher 
(2011) stated students in upper level undergraduate nutrition courses were better able to organize 
and coherently write article summaries using Thinking Maps®. 

Students in the middle school range in age from 11-15. This developmentally diverse 
group has its own characteristics that can make it difficult for teachers to be specific about 
instructional strategies that allow students to learn to the mastery level. A class may have 
students in the concrete operational level of development and require more structured, 
manipulative, concrete instructional strategies through which they can learn. Other students in 
the class may be in the formal operational level, who are able to think abstractly and may not 
require as many hands-on, manipulative-based activities in order to learn some concepts. 

Because of the cognitive diversity in the classroom, middle grades teachers must be able to 
access a multitude of concrete and abstract activities, experiences, and instructional resources to 
cover the differences within this age range. Using tools such as graphic organizers, advance 
organizers, and other organizational strategies could serve as a unifying strategy for each of these 
developmental levels to manipulate, organize, and utilize pertinent information. Tools that could 
potentially help both concrete and abstract learners may be Thinking Maps®. 



Concept Analysis Chart 


TOPIC: Studies Related to Thinking Maps Usage 


STUDY 

PURPOSE 

PARTICIPANTS 

DESIGN/ 

ANALYSIS 

OUTCOMES 

Ausubel, 

D.P. 

(1960) 

Test the 
hypothesis that 
learning can be 
facilitated by 
use of advance 
organizers 

120 senior 
undergraduate 
students, 78 
women & 32 men 
at Univ. of Illinois 

Quantitative: 

one-tailed 

test 

Assessment scores of 
students using advance 
organizers was higher 
than those not using 

Ball M.K. 
(1998) 

Test to see if 
use of TM's 
helps improve 
reading scores 

92 traditional & 
nontraditional Jr. 
college students in 
MS 

Quantitative: 

2-way 

MANCOVA; 

Qualitative: 

Questionnaire 

Treatment group 
outperformed control 
group in 5 of 7 variables 
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Brown, 

C.A. 

(2003) 

Test to see if 
elementary 
students could 
develop high- 
level reading 
comprehension 
tests in 

comparison to 
researcher's 
test using 

TM's 

20 4th grade 
students in 
suburban 

elementary school 

Quantitative: 
paired 
samples t- 
tests; 

Qualitative: 
evaluation of 
student¬ 
generated test 
items for 
cognitive 
level 

Paired samples t-tests 
showed positive 
correlations between 
tests; showed significant 
improvement answering 
test questions developed 
by peers; Qualitative 
evaluation of test items 
showed ability to 
generate higher-order 
questions 

Chang, 

Test the 

126 5th grade 

Quantitative: 

Map-correction strategy 

K.E., 

learning 

students in 

one-way 

worked significantly 

Sung, 

effects of 

elementary school 

ANCOVA of 

better than scaffold¬ 

Y.T., 

concept 

in Taipei, Taiwan, 

post-test 

fading & map- 

Chen, I.D. 
(2002) 

mapping 

60 girls & 66 boys 
divided into 4 
groups (3 
experimental, 1 
control) 

scores 

generation when post- 
hoc comparison using 
Bonferroni method used 

DeCecco, 

Examined if 

26 LD middle 

Quantitative: 

Test subjects in both 

V.M. & 

graphic 

school students in 

2-way 

groups scored 

Gleason, 

organizers 

2 Oregon middle 

condition x 

significantly higher on 

M.M. 

helped LD 

schools (1 low 

test ANOVA 

the post-tests (63% & 

(2002) 

middle school 

SES, 1 middle 

with repeated 

67%); Scheffe post hoc 


students attain 
& retain 
information 
from 

expository 

texts 

SES) 

measures, 
pre-test/post¬ 
test with 
control 
group; 
Woodcock 
Reading 
Mastery Test- 
Revised 

analysis showed 
significantly more 
relational knowledge 
with graphic organizer 
group than without 
graphic organizers 



Gallagher, 

M.L. 

(2011) 

Test to see if 

use of TM's 

improved 

students' 

ability to 

evaluate 

primary 

dietary 

literature 

Students in 
Advanced 

Nutrition & 
Metabolism 
classes at East 
Carolina 

University 

Qualitative: 
analysis of 
writing over 
each of 4 
semesters; 
informal 
interviews on 
perceptions 
of ability to 
write 

Use of TM's improved 
organization & clarity of 
writing over 4 semesters 
it was implemented; 
Students indicated they 
had better understanding 
of research process & 
were able to write about 
the 5 articles in more 
concise manner 

Gibbs, 

Test to see if 

3rd & 4th graders 

Quantitative: 

Ql: no evidence that 

S.L. 

reading 

in three schools in 

analysis of 

length of implementing 

(2009) 

achievement 

middle Tennessee 

test scores on 

TM's affected Rdg. 


of third & 

over 5 years of 

TCAP, 

Achievement, but 


fourth grade 

implementation 

independent 

showed increase in 


students who 

for 2 schools; 1 

sample t- 

student scores; Q2: no 


used TM's 

school with no 

tests; 

evidence that 


would improve 

treatment was 

used as a control 

Qualitative: 
surveys on 
perceptions 
of TM's by 
teachers 

implementing TM's 
affected Rdg. achieve¬ 
ment; Q3: well- 
established TM's 
program steadily 
improved scores vs. 
more recent 
implementation 

Hester, 

Provide 

One school in 

Quantitative: 

Test scores from first 

J.P., 

background 

Catawba Co., NC 

analysis of 

school demonstrated 

Owen, S., 

and 

with low test 

test scores, 

"outstanding" growth 

Piekarski, 

information on 

scores; also Tuttle 

no data in 

(no quantitative data); 

B„ & 
Hildebran, 
W. (1995) 

how Catawba 
Co. Schools 
developed & 
implemented 
the plan for 
integrating 

TM's into the 
curriculum 

Middle School 

article 

Tuttle MS had 
"upswing" in writing, 
reading, & other 
achievement test scores 
(no data) 
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Hickie, 

Test to see if 

70 students in 3 

Quantitative: 

Ql: results showed 

M.K. 

an association 

Title I elementary 

paired t-tests, 

significant differences 

(2006) 

exists between 

schools in 

3-factor 

b/w 2003 & 2005 


TM's 

Tennessee 

repeated 

Reading means, no 


instruction & 


measures 

significant diff. in Math 


student 


ANOVAs 

means; Q2: results 


achievement in 



showed no significant 


5 th grade 



diff. in school or gender, 


students in 



but the mean for 2005 


R/ELA& 



R/ELA scores was 8 


Math 



points higher; Q3: 
results showed no 
significant diff. in 
school, gender, or year 
for Math; Q4: results 
showed significant diff. 
b/w 2003 & 2005 

R/ELA means; Q5: 
results showed no 
significant diff. in 
interactions in school, 
year, or gender for Math 

Hyerle, 

1. To 

N/A 

Non- 

Base research & 

D.N. 

investigate the 


empirical 

development 

(1993) 

need for TM's. 


narrative with 



2. Define the 
maps & 
explore 
practical use. 

3. Present 

TM's as 


examples 



common 





visual 

language for 
organization, 
problem¬ 
solving, 
concept 
development, 

& thinking. 






Hyerle, 

D.N. 

( 2000 ) 


To aid readers 
in practical 
application of 


TM's 


Reports gains in 
scores in different 
schools across 
U.S. 


Quantitative: 
shows only 
gains in 
scores on 
assessments 


Fain Elem., Atlanta, 

GA: 1996, Reading 
scores increased 29- 
69% & math increased 
32-63% on GTBS; 
Friendship Valley Elem, 
Carroll County, MD: 
1996, Writing rose 27%, 
Language increased 
20%, Science rose 18% 
on MSPAP; Windemere 
Elem., West Orange 
County, FL: 1997 Math 
scores increased from 
79-92%, Reading 
increased 68-80% on 
Stanford-8; Waitz 
Elem., Mission, TX: 
1994 Math scores 
increased 41-76%, 
Reading increased 62- 
88% on TAAS 
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Hyerle, 

D.N. 

(2000), 

continued 

To aid readers 
in practical 
application of 
TM's 

Reports gains in 
scores in different 
schools across 

U.S. 

Quantitative: 
shows only 
gains in 
scores on 

assessments 

Claremont Elem., 
Catawba County, NC: 
1993-1996 Writing 
scores increased from 
33—68% at 4th grade on 
NC EOYT; Tuttle 

Middle, Catawba Co., 

NC: 1994-1996 Writing 
scores increased 43- 
73% at 8th grade on NC 
EOYT; Chadbourne 
Elem., Columbus Co., 
NC: 1993-1995 Writing 
scores increased from 
35-60% on NC EOYT; 
Morrisville Elem., 

Wake Co., NC: 1993- 
1996 Writing scores 
increased from 32-76% 
on NC EOYT 

Hyerle, 

D.N. 

(2000), 

continued 

To aid readers 
in practical 
application of 
TM's 

Reports gains in 
scores in different 
schools across 

U.S. 

Quantitative: 
shows only 
gains in 
scores on 

assessments 

Allen Elem., Cabarrus 
Co., NC: 1998 Reading 
scores increased from 
77-89%, Writing scores 
increased from 29-77%, 

& Math scores increased 
from 80-91% on NC 
EOYT 

Leary, 

S.F. 

(1999) 

To study the 
effects of TM's 

on 

achievement in 
math, reading, 

& language 

78 Fourth grade 
students 

Quantitative: 

3-way 

ANOVA 
analysis of 
Stanford -9 
test scores; 
Qualitative: 
interviews 
with teachers 

No significant 
difference between 
treatment & control on 
any of the variables 
(treatment vs. control, 
race, gender, previous 
achievement level) 



Russell, L. 
(2010) 

Explore the 
impact of 

TM's on 
reading comp, 
achievement 

199 Fourth grade 
& 199 Fifth grade 
students 

Quantitative: 

repeated 

measures 

ANOVA for 
variables 
socio¬ 
economic 
status, 
gender, 
ethnicity, & 
LEP; 

Qualitative: 
surveys for 
perceptions 

No significant 
difference in reading 
achievement during 4th 
& 5 th grade on the 

Texas Growth Index 
(socio-economic, ELP, 
gender, ethnicity); 
Repeated Measures 
ANOVA showed 
significant differences 
in TGI between 4th & 

5 th grade 

Sunseri, 

Study the 

71 Fourth grade 

Quantitative: 

No significant 

A.B. 

effects of 

students 

analysis of 

difference between 

(2011) 

using TM's to 


rubric scores; 

treatment & control; 


generate 


Qualitative: 

small difference in ELL 


expository 


interviews 

students' writing vs. 


writing with 


with students 

non-ELL students' 


non-ELL & 


& classroom 

writing 


ELL students 


observations 
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CHAPTER THREE 

METHODOLOGY 

This chapter describes the methodology and procedures used in this study. The purpose 
of the study was to determine if the use of Thinking Maps® has an effect on students’ academic 
achievement and to determine if teachers believe Thinking Maps® are effective in helping 
students learn science concepts at their grade level, with an assessment score of 800 or higher on 
the Georgia Criterion-Referenced Competency Tests. Additionally, the study investigated the 
potential relationship between teachers’ beliefs about Thinking Maps® and their effectiveness 
for improving students’ academic achievement in science. Previous research on Thinking 
Maps® has been conducted in elementary schools and colleges with mixed results. A few 
studies have been conducted in middle schools, some with students with learning disabilities. 
Much of the research conducted on Thinking Maps® has been conducted by Hyerle, the creator 
of Thinking Maps®, with the goal of establishing a connection between Thinking Maps® usage 
and improved test scores. 

The research questions for the study were: (1) To what extent do Thinking Maps® affect 
middle grades students’ academic achievement in science? (2) What are teachers’ perceptions 
about the use of Thinking Maps® with middle grades students? (3) What, if any, is the 
relationship between the use of Thinking Maps® and teachers’ perceptions and implementation 
in middle grades? Specifically, what changes or trends, if any, were evident in the science 
achievement of middle school students in classes that used Thinking Maps®? 



Participants 

This study took place in a suburban Title I school system comprised of eighteen schools 
in west central Georgia that serves nearly 10,500 students, with approximately 1100 staff 
members, in 2013 (Georgia Department of Education, 2013). There are eleven elementary 
schools, four middle schools with grades six through eight, and three high schools. The 
percentage of middle school students on free or reduced-price lunch in the 2012-2013 school 
year was 77.28% (Georgia Department of Education, 2013). The subjects for this study were 
students at the middle school level. During the 2012-2013 school year, the total population of 
middle school students in this system was 2667 (Georgia Department of Education, 2013). This 
means that over 2000 middle school students in this system are on free or reduced-priced lunch. 

During the 2012-2013 school year, females comprised 49.36% of the total middle school 
population, with black females representing 47.46% and white females comprising 45.39% of 
the population. Hispanic females made up less than 5% of the population. Male middle school 
students in this system during the same school year comprised 50.64% of the total population. 
White males comprised 48.45% of the population, while black males comprised 43.24% of the 
middle school population. The subgroup of Hispanic males comprised a little over 5% of the 
population (see Table 1) (Georgia Department of Education, 2012). 

Table 1 

School System Ethnicity and Gender Data, 2012-2013 


Gender 

Ethnicitv/Race 

Grade 06 

Grade 07 

Grade 08 

Total 

% 

Female 

Hispanic 

American 

16 

13 

19 

48 

4.13 


Indian 

3 

0 

0 

3 

0.26 



41 


Asian 

5 

2 

1 

8 

0.69 

Black 

180 

195 

176 

551 

47.46 

Pacific Islander 

0 

0 

0 

0 

0 

White 

193 

169 

165 

527 

45.39 

Two or More 

Races 

7 

9 

8 

24 

2.07 

Subtotal 

404 

388 

369 

1161 

49.36 

Hispanic 

22 

23 

15 

60 

5.04 

American 

Indian 

1 

0 

0 

1 

0.08 

Asian 

3 

3 

3 

9 

0.76 

Black 

168 

175 

172 

515 

43.24 

Pacific Islander 

0 

0 

0 

0 

0 

White 

190 

208 

179 

577 

48.45 

Two or More 

Races 

8 

8 

13 

29 

2.43 

Subtotal 

392 

417 

382 

1191 

50.64 

System Total 

796 

805 

751 

2352 



The student population during the 2013-2014 school year remained relatively static. The 
female population comprised 49.94% of the 2,352 total middle school students. Of the total, 
black females numbered 47.19% and white females numbered 43.61%. The number of Hispanic 
females increased to 59, which was 5% of the population. The population of males comprised 
50.06% of the total population. During this school year, the gap in the number of black and 
white males closed slightly, with black males comprising 44.27% of the middle school 
population, and white males comprising 46.05% of the population. Hispanic males also 
increased to comprise 6% of the total population (see Table 2) (Georgia Department of 


Education, 2013). 



Table 2 


School System Ethnicity and Gender Data, 2013-2014 


Gender 

Ethnicitv/Race 

Grade 06 

Grade 07 

Grade 08 

Total 

% 

Female 

Hispanic 

American 

23 

18 

18 

59 

5.03 


Indian 

0 

3 

1 

4 

0.34 


Asian 

8 

5 

2 

15 

1.28 


Black 

188 

175 

191 

554 

47.19 


Pacific Islander 

0 

0 

0 

0 

0 


White 

156 

195 

161 

512 

43.61 


Two or More 
Races 

13 

8 

9 

30 

2.56 


Subtotal 

388 

404 

382 

1174 

49.94 

Male 

Hispanic 

American 

23 

23 

25 

71 

6.03 


Indian 

2 

0 

1 

3 

0.25 


Asian 

4 

3 

5 

12 

1.02 


Black 

180 

170 

171 

521 

44.27 


Pacific Islander 

0 

0 

0 

0 

0 


White 

157 

187 

198 

542 

46.05 


Two or More 
Races 

10 

9 

9 

28 

2.39 


Subtotal 

376 

392 

409 

1177 

50.06 


System Total 

764 

796 

791 

2351 



The teacher demographics of the school system showed a large number of highly- 
qualified teachers with a Master’s degree or higher. The majority of teachers in the system were 
female and white, with an experience level of less than twenty years. 

The science department at one middle school in the district was selected for the current 
study. The demographics of this convenience sample showed a similar distribution to that of the 
district, where 92% of the science teachers or special education collaborative science teachers 
had a master’s degree or higher. Several of the teachers during the course of this study were 
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working on advanced degrees as well. Of the thirteen teachers who participated in this study, 
84.61% of them were female, and the same percentage was white, while 15.39% were black. 
Nearly 62% of the science teachers had greater than ten years’ experience (see Table 3). 


Table 3 

School System & Selected School PK-12 Teacher Demographics, 2012-2014 



2012-2013 

Svstem 

2013-2014 

Svstem 

Selected School 
Science Department* 

Certificate Level 

4-Yr Bachelor's 

242 

214 

1 

5-Yr Master's 

307 

302 

5 

6-Yr Specialist's 

104 

116 

5 

7-Yr Doctorate 

13 

19 

2 

Certified Personnel 

Professional 

654 

638 

13 

Provisional 

19 

19 

0 

Gender 

Female 

549 

528 

11 

Male 

124 

129 

2 

Race/Ethnicity 

Asian 

3 

3 

0 

Black 

197 

204 

2 

Hispanic 

11 

10 

0 

Multiracial 

11 

12 

0 

Native American 

0 

0 

0 

White 

451 

428 

11 

Years' Experience 

<1 

38 

37 

0 

1-10 

279 

249 

5 

11-20 

211 

223 

3 

21-30 

118 

123 

4 

>30 

27 

25 

1 



Note: Selected School Science Department retained the same personnel over the course of the study 


Instruments 

Teacher Survey. 

A researcher-created survey was used to determine teachers’ perceptions about the use of 
Thinking Maps® with middle grades students under the supervision of a university faculty 
member specializing in science education, and a panel of experts evaluated it prior to the study. 
Specifically, the survey was aimed at finding out the level of usage by teachers, if teachers 
believe Thinking Maps® help improve their students’ achievement in science, and if they believe 
Thinking Maps® are effective in helping students learn science concepts and contribute to 
students scoring in the “exceeds expectations” category (i.e., greater than 800) on the Georgia 
CRCT in science at their grade level. 

According to Trochim (2006), researchers use surveys to aid in the triangulation of data. 
Surveys serve to give the researcher either a wider view of the population’s perceptions, beliefs, 
or views, or a narrower view, which can then lead to more in-depth probing and information 
access. Surveys may yield better participation. Respondents may be quicker to complete a 
survey and be more honest, give short, concise answers, and may provide detail to open-ended 
questions due to greater privacy. Using surveys, the researcher can better understand teachers’ 
perceptions of Thinking Maps®. Surveys of teachers were conducted to evaluate their use of 
Thinking Maps® in their classes (see Appendix). 

Georgia Criterion-Referenced Competency Test (CRCT) science test scores for 2012- 
2013 and 2013-2014 were obtained and analyzed, along with usage and beliefs data from the 
survey, using the IBM Statistical Package for the Social Sciences version 24 for Windows 



45 


(SPSS) and the SAS Institute’s Statistics Analysis Software (SAS/STAT®) statistical analysis 
software packages. 

Research question one: To what extent do Thinking Maps® affect middle grades 
students’ academic achievement in science? 

Research question two: What are teachers’ perceptions about the use of Thinking Maps® 
with middle grades students? 

Research question three: What, if any, is the relationship between the use of Thinking 
Maps® and teachers’ perception and implementation in the middle grades? It considered the 
existence of a relationship between the use of Thinking Maps® and changes, or trends, in the 
science achievement of students in middle grades who used Thinking Maps®. 

According to the Standards for Educational and Psychological Testing (1999), validity is 
defined as “the degree to which evidence and theory support the interpretations of test scores 
entailed by proposed use of tests” (p. 9). The Georgia Department of Education developed the 
Criterion-Referenced Competency Tests with validity by using a multi-tiered process that began 
with the curriculum for each grade level, in which teams of educators established a test blueprint 
and test specifications for how the standards would be assessed. From the blueprint and 
specifications, the content domain specifications were developed to separate specific standards 
into strands or domains. Then, test item specifications were generated to give the item format, 
content scope limits, and cognitive complexity for the test items for the test items that were to be 
written. When complete, this working document is then called CRCT Content Descriptions. At 
this point, professional assessment specialists wrote test items as described in the content 
descriptions and in the content weights for each domain. Teams of educators reviewed the test 
items for curriculum alignment, potential bias, cultural or gender sensitivity, and suitability, then 



revised, edited, or rejected items as needed. Then the items were placed on field tests or as items 


on operational tests. Once items were field tested, educators conducted data analyses to see if 
the items had any biases and revised, edited, or rejected test questions again. The questions that 
were accepted were then placed on an item test bank for future inclusion on an operational test. 

The standards of testing, as established by the American Educational Research 
Association, the American Psychological Association, and the National Council on Measurement 
in Education, defined reliability as “the degree to which test scores for a group of test takers are 
consistent over repeated applications of a measurement procedure and hence are inferred to be 
dependable, and repeatable for an individual test taker; the degree to which scores are free of 
errors of measurement for a given group” (p. 180). Reliability coefficients are estimated values 
that fall between 0 and 1 (Trochim, 2006). The closer to 1 the coefficient falls, the more reliable 
the instrument. The reliability for the both the 2012-2013 and 2013-2014 CRCT was 0.92 for 
sixth grade, 0.94 for seventh grade, and 0.92 for eighth grade (see Table 4). 


Table 4 


CRCT Reliability Coefficients for Science by Grade, 2013 and 2014 


Grade 2013 2014 



Alpha 

SEM 

Alpha 

SEM 

6 

0.92 

3.28 

0.92 

3.31 

7 

0.94 

3.01 

0.94 

3.00 

8 

0.92 

3.28 

0.92 

3.24 


Note. Alpha = Cronbach’s alpha; SbM = Standard brror ol Measurement 
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Research Design 

This research study utilized a mixed-methods design. It used both primary and secondary 
data sources. The primary data consisted of Georgia Criterion-Referenced Competency Tests 
science test scores, and the secondary sources of data included survey data and interview data 
from teachers on their perceptions about the use of Thinking Maps® to aid students in learning 
science content. According to the National Institutes of Health Office of Behavioral and Social 
Sciences Research (2015), mixed-methods research is one “employing rigorous quantitative 
research assessing magnitude and frequency of constructs and rigorous qualitative research 
exploring the meaning and understanding of constructs” (p. 4). Descriptive research involves 
gathering data that describe events and then organizes and describes the collected data (Glass & 
Hopkins, 1984 ). This study used a concurrent triangulation mixed-methods research design 
(Creswell, 2003). The quantitative data were from two sequential school years, 2012-2013 and 
2013-2014. However, these data were not analyzed prior to the collection of the survey and 
interview data. Rather, the quantitative and qualitative data were analyzed concurrently to 
triangulate the data. 

Quantitative data for this study consisted of CRCT scores for the 6 th , 7 th , and 8 th grade 
students at the selected school for the 2012-2013 and 2013-2014 school years. These data were 
collected by the school at the end of each school year; however, scores for this study were 
obtained by the researcher as a part of the research process following approval by the 
Institutional Review Board (IRB). This study was classified as exempt and permission to begin 
data collection was received on September 6 , 2016. Thinking Maps® usage data from the 
teacher survey were quantified for the purpose of analysis. All physical copies of data and 
materials, from notes, printouts, SPSS/SAS output, surveys, and portable storage devices, will be 
stored in a locked file cabinet off campus with the researcher for a minimum of three years. 



Qualitative data for this study came from the open-ended questions on the teacher survey. 
These data were used to triangulate and expand upon the quantitative analyses. 

Data Collection 

Middle grades science teachers were surveyed regarding their perceptions and use of 
Thinking Maps® (see Appendix). The researcher developed the survey with the direct assistance 
of a faculty member from the researcher’s university who specialized in science education. 
Faculty at the selected school were apprised of the study and informed that participation was 
voluntary prior to distribution of the survey. When it was determined that only the science 
department and those special education teachers who taught science should be the focus of the 
survey, those teachers were informed verbally prior to dissemination of the surveys. All six of 
the regular education science teachers, six special education collaborative science teachers, and 
one gifted education teacher were surveyed at the selected school, and all 13 members of the 
science department participated. 

Science scores for students from the 2012-2013 and 2013-2014 administrations of the 
Georgia Criterion-Referenced Competency Tests were obtained from the assistant principal for 
instruction at the selected school from the copy received from the Georgia Department of 
Education that included only the students’ student number and achievement score. The 
researcher developed a spreadsheet of the scores in preparation for analysis using the IBM 
Statistical Package for the Social Sciences version 24 (SPSS) and the SAS Institute’s Statistics 
Analysis Software (SAS/STAT®) statistical analysis software packages to see if there was a 
difference in the scores of those using Thinking Maps® systematically and consistently and 
those not using Thinking Maps® systematically and consistently. 



49 


Procedures 

In this study, teachers were surveyed on their perceptions and beliefs about Thinking 
Maps® and if they believe Thinking Maps® increase their students’ science achievement. 
Teachers were surveyed regarding the amount of use of Thinking Maps® in their classes, which 
maps they used the most, if they believe Thinking Maps® are effective and help students learn 
science concepts, which maps they believe are least effective, how Thinking Maps® are used in 
their classes, and if they believe Thinking Map® achieve the projected outcomes (i.e., improved 
science scores on the CRCT). 

Data Analysis 

The data collected from teacher surveys were analyzed for the subject area, grade, 
number of years using Thinking Maps®, and if teachers believed Thinking Maps® help students 
learn science. Two of the questions had both a finite answer and an open-ended part. Four of 
the questions were open-ended and were analyzed and the responses categorized using open 
coding, a process by which qualitative data is broken down and categorized by certain behaviors 
to discover what it means (Benaquisto, 2008). Survey data from teachers participating in the 
study were analyzed for the number of times Thinking Maps® were used in a day/week and 
classes, which were used the most, which were the least/most effective, which the teacher 
believed helps students learn better, and if the teacher believed Thinking Maps® achieve the 
intended outcome. Two of the questions included an open-ended part, and one question was 
completely open-ended. Open coding was used to analyze and categorize the open-ended 


questions. 



Students who received instruction using Thinking Maps® in a systematic and consistent 
manner were included in the group using Thinking Maps®, and those students who did not 
receive instruction with Thinking Maps® in a systematic and concise manner were included in 
the non-usage group. The Thinking Maps® usage was determined by the teacher surveys. Using 
Thinking Maps® at least 3 days a week and at least once per class was considered systematic and 
consistent usage, any usage less frequent than this was considered non-systematic and non- 
consistent usage. 

This study also focused on the teachers’ perceptions and beliefs about the Thinking 
Maps® and their use at the 6-8 grade levels in science. Teacher perception surveys and the 
analysis of CRCT scores in science were used for the two groups to determine if there was a 
relationship between the achievement scores and the teachers’ perceptions and beliefs as to the 
efficacy of the Thinking Maps® program. 

Lastly, science scores for students from the 2012-2013 and 2013-2014 administrations of 
the Georgia Criterion-Referenced Competency Tests were analyzed using SPSS and SAS to 
determine if there was a difference in the scores of those using Thinking Maps® and those not 
using Thinking Maps®. Science achievement scores were analyzed using appropriate statistical 
analysis using SPSS and SAS/STAT depending on the skewness and kurtosis values. Test scores 
were compared at their respective grade levels for consistent and systematic use of Thinking 
Maps® and those without consistent and systematic use of Thinking Maps®. Then, aggregated 
scores in each usage group were analyzed along with the teacher surveys to determine if a 
relationship existed between use of Thinking Maps® and teacher perceptions and 
implementation of Thinking Maps®. A relationship that correlates the teacher’s belief that 
Thinking Maps® used consistently and systematically help students learn science with favorable 
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test scores could make a strong case for evidence that Thinking Maps® helped increase students’ 
science achievement. Conversely, a relationship that did not correlate the teacher’s belief that 
Thinking Maps® not used consistently and systematically help students learn science with 
unfavorable test scores could make a strong case for evidence that Thinking Maps® do not help 
students’ science achievement. Because the CRCT scores are quantitative data and the teacher 
surveys include qualitative data, a relationship may be determined by a teacher believing in and 
consistently using Thinking Maps® and having high achievement scores on the CRCT. The 
opposite could be true as well. The results of an analysis of variance in the test scores may 
indicate the relationship between high scores and usage. 

Test score data for students’ science scores from the 2012-2013 and 2013-2014 
administrations of the CRCT were analyzed first to determine if a statistically significant 
difference in CRCT scores of classes using Thinking Maps® and those classes not using 
Thinking Maps®. Then, scores of those classes using Thinking Maps® were analyzed using 
non-parametric, one-way ANOVA with the use of Thinking Maps® as the independent variable, 
CRCT scores as the dependent variable. This paired with the survey data will help to determine 
if there is a relationship between the use of Thinking Maps® and teachers’ implementation and 
perceptions, as measured by CRCT science scores. 

Summary 

This study examined the effects of Thinking Maps® on the achievement of middle school 
students in science. Data consisted of surveys of teachers on their perceptions and 
implementation of Thinking Maps®, if they believed the maps help their students learn, and their 
use of Thinking Maps® in their classrooms. Science achievement scores from the 2013 and 



2014 spring administrations of the Georgia Criterion-Referenced Competency Tests (CRCT) 


were analyzed using a non-parametric, one-way ANOVA using the SPSS and SAS/STAT data 
analysis software packages. The results were analyzed to determine if there is a relationship 
between the use of Thinking Map® and teachers’ beliefs and perceptions, as measured by CRCT 
science achievement scores. 


CHAPTER FOUR 

RESULTS 

The purpose of this study was to determine if the use of Thinking Maps® has an effect on 
students’ academic achievement and to determine if teachers believe Thinking Maps® are 
effective in helping students learn science concepts at their grade level, with an assessment score 
of 800 or higher on the Georgia Criterion-Referenced Competency Tests. Additionally, the 
study investigated the potential relationship between teachers’ beliefs about Thinking Maps® 
and their effectiveness for improving students’ academic achievement in science. The research 
questions driving this were: (1) - To what extent do Thinking Maps® affect middle grades 
students’ academic achievement in science? (2) - What are teachers’ perceptions about the use 
of Thinking Maps® with middle grades students? (3) - What, if any, is the relationship between 
the use of Thinking Maps® and teachers’ perception and implementation in middle grades? 
Specifically, what changes or trends, if any, were evident in the science achievement of middle 
school students in classes that used Thinking Maps®? 
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To answer the first question, standardized test scores from the Georgia Criterion- 
Referenced Competency Tests (CRCT) in science for grades six through eight were analyzed for 
differences in the scores based on the usage or non-usage of Thinking Maps® in the science 
classes. Three teachers, one at each level (6 th , 7 th , and 8 th grades), used Thinking Maps® 
frequently in their practice, and three teachers did not use Thinking Maps® in their practice. 

The second question investigated teachers’ perceptions about using Thinking Maps® 
with their middle grades science classes. This information was gathered by surveying science 
teachers. The survey asked in which branch of science, earth; life; or physical, the teacher used 
or did not use Thinking Maps®, if they received training in Thinking Maps®, if they believe 
Thinking Maps® helps their students learn science, and which Thinking Maps® they believe are 
most effective and least effective. Teachers were also asked, through the survey, which 
Thinking Maps® they used the most if they used them, how often they used Thinking Maps® if 
they used them, how they used them in their classes if they used them, and if they believe 
Thinking Maps® achieve their designated outcome in learning science content. Additionally, 
teachers’ demographic information was collected. 

The third research question considered the existence of a relationship between the use of 
Thinking Maps® and the teachers’ perceptions of the effectiveness of Thinking Maps®, and if 
changes or trends in the achievement of students in middle grades were evident for those who 
used Thinking Maps®. 

This chapter is divided into three sections: analysis of student test score data, analysis of 
teacher survey data, and exploration of a possible relationship between the use of Thinking 
Maps® and students’ test scores. The first section analyzes demographic data and the science 
test scores from the 2012-2013 and 2013-2014 administrations of the Georgia Criterion- 



Referenced Competency Tests (CRCT) for non-usage or usage of Thinking Maps® across the 


three grade levels. The second section analyzes responses from science teachers regarding their 
use and perceptions of Thinking Maps®. It includes demographic data, which may be used to 
determine if age or experience play a factor in the use or perception of Thinking Maps®. The 
third section will discuss if there is a relationship between the use of Thinking Maps® and 
students’ test scores. 


Data Analysis 

Descriptive Statistics, 2012-2013 

Sixth, seventh, and eighth grade students from a Title I middle school located in a 
suburban school district in middle Georgia provided the sample and focus of this study. Scores 
for 702 students for the 2012-2013 were separated into two groups: (a) Students who did not use 
Thinking Maps® instruction in the classroom and (b) Students who did use Thinking Maps® 
instruction in the classroom. Non-parametric tests were conducted. 

Table 1 provides information on descriptive statistics of CRCT scores based on usage or 
non-usage of Thinking Maps®. For the 2012-2013 CRCT science test administration overall 
scores, there were a total of 702 students, with 349 students in the Non-Thinking Maps® or non¬ 
usage instructional group, which was 49.72% of the total students tested, and 353 students 
comprising the Thinking Maps® or usage instructional group, which was 50.28% of the total 
students tested. In the non-usage group, the mean CRCT score was 825.94 (n=349, SD=36.500), 
and the mean CRCT score for the usage group was 822.98 (n=353, SD=33.202) (see Table 1). 
The difference in means was 2.96. Overall, CRCT scores were higher for the non-usage group, 
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or those who did not use Thinking Maps®, than the usage group or those who used Thinking 
Maps®. 

There were 130 sixth grade students, or 50.39% of the total number of sixth graders, who 
were in the non-usage group, and 128 sixth grade students, or 49.61%, who were in the usage 
group. Together the sixth grade comprised 36.75% of the total number of students tested on the 
science CRCT. Of the 224 seventh grade students tested, 108 of them, or 48.21%, were in the 
non-usage group, while 116, or 51.79%, were in the usage group. Seventh graders comprised 
31.91% of the total number of students tested. There were 111 eighth grade students, or 50.45% 
of the total number of eighth graders, in the non-usage group, and 109 students, or 49.55% of the 
students tested, in the usage group. The eighth graders made up 31.34% of the total number of 
students tested on the science CRCT (see Table 2). 

Table 2 presents descriptive statistics of the CRCT science test scores by grade level. For 
the 2012-2013 CRCT test administration, there was a total of 258 sixth grade students, 
comprising 36.75% of the total students being tested. The mean CRCT score for the sixth grade 
was 821.54 (m=258, SD=30.733). There was a total of 224 seventh grade students, making up 
31.91% of the tested population. The mean CRCT score for the seventh grade was 839.00 
(m=224, SD=34.752). There was a total of 220 eighth grade students, comprising 31.34% of the 
tested population. The mean CRCT score for the eighth grade was 813.05 (m=220, SD=34.651). 
The difference between the means measured 17.46 between the sixth and seventh grades and 
25.95 between the seventh and eighth grades. The average difference between the means was 
21.705 (see Table 2). This showed the seventh grade having a marked difference in means from 


the sixth and eighth grade groups. 



The National Institute of Standards and Technology at the U.S. Department of Commerce 
defines a confidence interval as a range that “would bracket the true population parameter in 
approximately 95% of the cases” if sampled on multiple occasions (NIST, 2013). This means 
that if we sampled the population in a different year, each grade would yield achievement scores 
that fell within the confidence interval for the same teachers. The sixth grade results show a 
mean of 840.13 (n=130, SD=28.611) for the non-usage group, and a mean of 802.66 (n=128, 
SD=19.156) for the usage group, the non-usage group showing a 37.47 point difference. The 
difference between the upper bound and lower bound scores in the 95% confidence interval for 
the sixth grade was 9.93 for the non-usage group and 6.70 for the usage group. The seventh 
grade non-usage group scored a mean of 840.65 (n=108, SD=37.545), and the usage group had a 
mean of 837.46 (n=116, SD=32.022). The mean scores for the non-usage and usage groups were 
closer in the seventh grade, but the non-usage group still scored higher than the usage group. 

The difference in the seventh grade confidence intervals was 14.32 for the non-usage group and 
11.78 for the usage group, a difference of 2.54. The non-usage group in the eighth grade scored 
a mean of 795.00 (n=lll, SD=21.614), and the usage group scored a mean of 831.42 (n=109, 
SD=35.824). Contrary to the other two grades, the eighth graders who used Thinking Maps® 
scored an average of 36.42 points higher than the non-usage group. The difference in the eighth 
grade confidence intervals was 8.14 for the non-usage group and 13.60 for the usage group, a 
difference of 5.46. The mean CRCT scores were approximately the same for the sixth and 
seventh graders who did not use Thinking Maps®, but the eighth grade students who did not use 
Thinking Maps® scored much lower than their usage counterparts, with a mean of 795.00 (see 


Table 2). 
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Table 1 

Descriptive Statistics, Usage & Non-Usage of Thinking Maps®, 2012-2013 
Science CRCT 


Group Statistics 

SE 



N 

Mean 

SD 

Mean 

Skewness 

Kurtosis 

Score Non-usage 

349 

825.94 

36.500 

1.954 

0.342 

-0.202 

Usage 

353 

822.98 

33.202 

1.767 

0.778 

0.895 


Note. N = sample size; SD = standard deviation; Sh = standard error 


Table 2 


Descriptive Statistics by Non-Usage and Usage, 2012-2013 Scores by Grade Level 




6th Grade 



7th Grade 



8 th Grade 



Non- 



Non- 



Non- 




Usage 

Usage 

Total 

Usage 

Usage 

Total 

Usage 

Usage 

Total 

N 

130 

128 

258 

108 

116 

224 

111 

109 

220 

% Total 
Tested 

50.39 

49.61 

100.00 

48.21 

51.79 

100.00 

50.45 

49.55 

100.00 

Mean 

840.13 

802.66 

821.54 

840.65 

837.46 

839.00 

795.00 

831.42 

813.05 

SD 

28.611 

19.156 

30.733 

37.545 

32.022 

34.752 

21.614 

35.824 

34.651 

SE 

2.509 

1.693 

1.913 

3.613 

2.973 

2.322 

2.052 

3.431 

2.336 

Skewness 

0.175 

0.015 


0.208 

0.077 


0.241 

0.929 


SE 

0.212 

0.214 


0.233 

0.225 


0.229 

0.231 


Kurtosis 

0.101 

-.760 


-0.116 

-0.419 


-0.442 

1.59 


SE 

0.422 

0.425 


0.461 

0.446 


0.455 

0.459 


95% Cl 










Upper 

Bound 

845.10 

806.01 


847.81 

843.35 


799.07 

838.22 


Lower 

Bound 

835.17 

799.31 


833.49 

831.57 


790.93 

824.62 



Non-Usage 

Total 349 

Usage Total 353 

Note. N = sample size; SD = standard deviation; Sb = standard error; Cl = confidence interval. 



Tests of Normality, 2012-2013 

According to Field (2009), skewness is defined as the “lack of symmetry” in the 

distribution of variables. Data was aggregated by non-usage/usage and then by grade level. The 
histogram for the non-usage group showed a positive or right-skewed non-normal distribution 
with a skewness of .342 (SE = .131), while the histogram for the usage group also showed a 
positive or right-skewed non-normal distribution of .778 (SE = .130). Additionally, kurtosis is 
defined as “the degree to which scores cluster at the ends of the distribution” (Field, 2009). 
Westfall (2014) stated kurtosis “measures the departure from normality.” The non-usage group 
had a negative kurtosis of -.202 (SE = .260). The usage group had a positive kurtosis of .895 (SE 
= .259). From this, we see the magnitude of kurtosis is higher for the usage group. This 
indicated that both distributions vary from normal distribution (see Figure 1 and 2). The 
Shapiro-Wilk test was conducted to test whether the CRCT test scores for the non-usage and 
usage groups departed from normality to a statistically significant degree. A significant test (p < 
.05) indicates the CRCT scores depart from normality, whereas a non-significant test (p > .05) 
indicates the scores follow a normal distribution. The Shapiro-Wilk test for each of the non¬ 
usage and usage groups, was significant, indicating the scores departed from normality (all p’s < 
.05), which gives evidence that we can reject the null hypothesis (see Table 3). Levene’s test for 
homogeneity of variances for the non-usage group versus usage group was used to determine that 
variances in the data set were equal when the data exhibited a non-normal distribution. Tests for 
homogeneity of variances showed F(l,700) = 6.230, p = .013, which was not statistically 
significant because p < .05, indicating that the differences in CRCT science test scores across the 
grade levels is equal to a statistically significant degree, and it can be concluded there are 
unequal variances in the sixth, seventh, and eighth grades (see Table 4). 



Figure 1 

Histogram of Science CRCT Scores, Non-Usage, 2012-2013 


59 


Histogram 


for Usage_Nonusage= Nonusage 



Figure 2 

Histogram of Science CRCT Scores, Usage, 2012-2013 
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for Usage_Nonusage= Usage 
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Table 3 


Tests of Normality for Non-Usage and Usage, 2012-2013 



Kolmosorov-Smirnov a 


Shaniro-Wilk 



Statistic 

df 

Sig- 

Statistic 

df 

Sig- 

Non-Usage 

0.061 

349 

0.003 

0.985 

349 

0.001 

Usage 

a. Lillielors Si 

0.070 

emlicance Correction 

353 

0.000 

0.966 

353 

0.000 


Table 4 

Levene’s Test for Homogeneity of Variance, Non-Usage and Usage, 2012-2013 


Score Based on Mean 

6.230 

1 700 

0.013 

Based on Median 

6.443 

1 700 

0.011 

Based on Median and 

with adjusted df 

Based on trimmed 

6.443 

1 699.885 

0.011 

Mean 

6.509 

1 700 

0.011 


The histogram for the sixth grade reported a mean of 821.54 (n=258, SD=30.733) and 
showed a skewness of .508 with a standard error of .152, and a platykurtic kurtosis of .035 with a 
standard error of .302, showing a non-normal distribution (see Figure 3). The histogram for the 
seventh grade reported a mean of 839.00 (n=224, SD=34.752) and showed a skewness of .178 
with a standard error of .163, and a kurtosis of -.159, also platykurtic, with a standard error of 
.324, showing a non-normal distribution (see Figure 4). The histogram for the eighth grade 
reported a mean of 813.05 (n=220, SD=34.651) and showed a skewness of 1.072 with a standard 
error of .164, and a kurtosis of 2.057, also platykurtic, with a standard error of .327, showing a 
non-normal distribution (see Figure 5). Normality was supported by the Shapiro-Wilk test for 
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non-usage and usage for the sixth and seventh grades and only the non-usage group for the 
eighth grade because p > .05. Normality was supported by the Shapiro-Wilk test for only the 
usage group for the eighth grade, because p < .05 (see Table 5). Levene’s test for homogeneity 
of variances for grade level versus non-usage/usage was used to determine that variances in the 
data set were equal when the data exhibited a non-normal distribution. Tests for homogeneity of 
variances showed F{2,699) = 1.819, p = .0163, which was statistically significant because p > 
.05, indicating there is a difference between the grade levels to a statistically significant degree 
because the test is significant (see Table 6). We reject the null hypothesis, which states “there is 
no statistically significant difference in the variances of CRCT science test scores between grade 
levels,” and it can be concluded there are unequal variances in the sixth, seventh, and eighth 
grades. 

Figure 3 

Histogram of Sixth Grade CRCT Scores, 2012-2013 


Histogram 
for Grade= 6th grade 



Score 


























Figure 4 

Histogram of Seventh Grade CRCT Scores, 2012-2013 


Histogram 
for Grade= 7th grade 



Figure 5 

Histogram of Eighth Grade CRCT Scores, 2012-2013 
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Histogram 


for Grade= 8th grade 



Table 5 

Tests of Normality for Sixth, Seventh, and Eighth Grade, 2012-2013 


Tests of Normality 

K o 1 m o go ro v - S in i r n o v a Shapiro-Wilk 

Non-Usage 


Level 

Usage 

Statistic 

df 

Sig- 

Statistic 

df 

Sig. 

6 

Non-Usage 

.055 

130 

.200 

.988 

130 

.308 


Usage 

.088 

128 

.016 

.984 

128 

.125 

7 

Non-Usage 

.058 

108 

.200 

.986 

108 

.324 


Usage 

.058 

116 

.200 

.988 

116 

.387 

8 

Non-Usage 

.078 

111 

.097 

.982 

111 

.137 


Usage 

.103 

109 

.006 

.950 

109 

.000 


~a. Lilliefors Significance Correction 

























Table 6 


Levene’s Test of Homogeneity of Variance, Grades 6, 7, and 8, 2012-2013 



Levene Statistic 

dfl 

df2 

Sig. 

Score Based on Mean 

1.819 

2 

699 

0.163 

Based on Median 

1.816 

2 

699 

0.163 

Based on Median and 
with adjusted df 

1.816 

2 

677.064 

0.163 

Based on trimmed Mean 

1.882 

2 

699 

0.153 


Transformations, 2012-2013 

The Shapiro-Wilk and Kolmogorov-Smimov normality tests indicated the scores for the 
non-usage of Thinking Maps® and the usage of Thinking Maps® groups were significant for 
both groups, so there was a need to transform the scores to reach a normal distribution. The 
scores did not follow a normal distribution even after transformations. However, because of the 
non-normal distribution shown in Figures 1 and 2, it was necessary to perform a non-parametric 
version of the independent Mcst. An examination of skewness (sixth = .508; seventh = .178; 
eighth = 1.072), kurtosis (sixth = .035; seventh = -.159; eighth = 2.057), and standard deviations 
(sixth = 30.733; seventh = 34.752; eighth = 34.651) for the three grade levels and a visual 
examination of the histograms revealed the scores departed from normal, indicating a need to 
transform the scores to reduce the variance in the data. In addition, the Shapiro-Wilk and 
Kolmogorov-Smirnov normality tests were significant, which further warranted a need to apply 
the appropriate transformations on the CRCT scores so they more approximately followed a 
normal distribution and fulfilled the assumption of parametric statistical analysis. Therefore, a 


log e transformation was completed for the three grade levels. Because there were two levels, 
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non-usage and usage, a Mann-Whitney test was performed. Also, because there were three grade 
levels, a non-parametric version of the ANOVA was performed. 

Non-Parametric Analysis, 2012-2013 

The CRCT scores followed a non-normal distribution. The Kruskal-Wallis test is a rank- 

based non-parametric test, also known as the rank-based one-way ANOVA, that determines 
whether the CRCT scores that have been sampled from the entire population of CRCT scores 
originate from the same distribution. The independent variables were categorical, having three 
or more categories (sixth, seventh, and eighth grades), and the dependent variable was not 
normally distributed among the CRCT scores. The statistical tests constructed for data following 
a non-normal distribution needed to be ranked because these tests make fewer assumptions about 
the data, unlike the parametric tests. Therefore, the CRCT scores were first converted to ranked 
scores and then used in the Kruskal-Wallis test. The mean CRCT scores for the sixth, seventh, 
and eighth grades were 821.54, 838.45, and 813.05, respectively. 

All non-parametric statistical analysis was conducted in SAS. The PROC NPAR1WAY 
is an in-built procedure in the SAS library to perform non-parametric analyses. PROC 
NPAR1WAY provides a summary of Wilcoxon-ranked scores for CRCT scores by each grade 
level, and the Kruskal-Wallis test. The Rruskal-Wallis tests the hypothesis that samples come 
from identical populations, that is the mean ranks of the groups are the same. The CRCT scores 
from the sixth, seventh, and eighth grades have equal-ranked mean scores. Therefore, the 
Kruskal-Wallis test was significant when the ranked mean CRCT scores between the sixth, 
seventh, and eighth grades are different from each other to a statistically significant degree, 
indicating that the CRCT scores of the three grade levels come from different populations. Table 



7 shows that the mean CRCT-ranked scores were different from each other with seventh grade 
having the highest CRCT-ranked mean score with 437.16, followed by the sixth grade with 
336.88, and eighth grade with 280.22. 

The Kruskal-Wallis test was significant (y~ = 68.5175, p < .01). Grade level had an effect 
on the CRCT scores to a statistically significant degree (see Table 8). There is sufficient 
evidence to reject the claim that CRCT-ranked mean scores from the three grade levels have 
equal-ranked mean scores. At least one of the ranked mean CRCT scores in one grade level 
appeared to be different from the other two grade levels. The Wilcoxon rank-sum test also 
depicted the same picture as the Kruskal-Wallis test where not only the sum of scores was above 
the standard deviation of Ho but also the ranked mean score was different across grade levels, 
with the seventh grade having the highest CRCT-ranked mean score of 437.16. Therefore, the 
CRCT mean scores as well as the ranked mean scores was influenced most by the seventh grade, 
followed by the sixth grade and the eighth grade. That is, the seventh grade explains the most 
variance in CRCT scores when compared to the sixth and eighth grades. The distribution of 
CRCT-ranked sum scores in the box plot graphically depicts large differences in the lower 
quartile, middle quartile, and upper quartile scores based on grade levels (see Figure 6). The 
frequencies bar graph shows that seventh grade had a higher frequency of CRCT scores above 
the median than below the median, followed by the sixth and eighth grades (see Figure 7). 

Table 7 


NPAR1 WAY Wilcoxon Results by Grade Levels, 2012-2013 
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Wilcoxon Scores (Rank Sums) for Variable CRCT 


Classified by Variable Grade 


Grade 

N 

Sum of 
Scores 

Expected 
Under H 0 

Std Dev 
Under H 0 

Mean Score 

8 

220 

61649.00 

77220.00 

2487.63091 

280.222727 

7 

223 

97486.00 

78273.00 

2496.71199 

437.156951 

6 

258 

86916.00 

90558.00 

2585.31688 

336.883721 


Average scores were used for ties. 


Table 8 

Kruskal-Wallis One-Way Test Results by Grade Levels, 2012-2013 


Kruskal-Wallis Test 

Chi-Square 

68.5175 

DF 

2 

Pr > Chi-Square 

<.0001 



Figure 6 

Wilcoxon Box Plot Scores for CRCT by Grade Level, 2012-2013 


Distribution of Wilcoxon Scores for CRCT 



8 7 6 

Grade 


Figure 7 

Frequencies Above & Below the Median CRCT Scores by Grade Level, 2012-2013 
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Frequencies Above and Below the Overall Median for CRCT 



i -1-r 

8 7 6 

Grade 


■ Not Above the Median E Above the Median 


The asymptotic p-value for the Kolmogorov-Smimov test was 3.9571, which indicates a 
rejection of the null hypothesis that CRCT score distributions were identical for the three grade 
levels (see Table 9). The empirical distribution of the CRCT scores were derived from the true 
distribution of CRCT scores. The empirical distribution graph of the CRCT scores showed the 
proportion of scores falling in the CRCT score ranges (see Table 10). The graph shows that the 
variation in the distribution of CRCT scores across the three grade levels was different but the 
magnitude of difference was much more in the seventh grade when compared to the sixth and 
eighth grades. The CRCT score ranges are on the X-axis and the proportion of CRCT scores fell 
in a range marked on the Y-axis. The graph shows that there was more variation in the seventh 
grade curve of CRCT score distribution, especially after the curve cross the 850 mark (indicated 













by a steeper curve) when compared to the sixth and eighth grades and a graduation variation 
the eighth grade curve (see Figure 8). 

Table 9 

Kolmogorov-Smirnov Asymptotic Statistics, 2012-2013 

KS 0.14946 KSa 3.957176 


Table 10 

Kolmogorov-Smirnov Empirical Distribution Test by Grade Level, 2012-2013 


Kolmogorov-Smirnov Test for Variable CRCT 
Classified by Variable Grade 


Grade N EDF at Maximum 


Deviation from Mean at 
Maximum 


8 

220 

0.672727 

2.276314 

7 

223 

0.309417 

-3.133597 

6 

258 

0.569767 

0.811299 

Total 

701 

0.519258 



Note. bDh = Empirical Distribution Function 


Figure 8 

Empirical Distribution Graph for CRCT Scores by Grade Level, 2012-2013 
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Empirical Distribution for CRCT 



The Wilcoxon two-sample test is the non-parametric version of a t-test and is used to 
determine whether two samples were derived from a population having the same distribution. 
The two-sample test was utilized to determine the influence of non-usage and usage of Thinking 
Maps® on the variation in CRCT scores. The mean CRCT score for non-usage and usage 
groups was 822.98 and 825.94, respectively. The Wilcoxon mean-ranked CRCT scores for non¬ 
usage (coded as 1) was 361.02 and for usage (coded as 2) was 342.09 (see Table 11). 

The Wilcoxon two-sample test was not significant, indicating that Thinking Maps® did 
not influence the variation in CRCT scores. There is not sufficient evidence to reject the claim 
that the CRCT-ranked mean scores for the non-usage and usage groups had equal-ranked mean 
scores (see Tables 12 and 13). The distribution of CRCT-ranked sum scores in the box plot 
graphically does not depict large differences in the lower quartile, middle quartile, and upper 
quartile scores based on non-usage and usage levels. The box plot reveals the same where the 














distribution for non-usage and usage are similar (see Figure 9). The bar chart shows that there is 
approximately equal distribution of the median CRCT scores above and below the median for 
both non-usage and usage groups (see Figure 10). 


Table 11 

Wilcoxon Test Results by Non-Usage and Usage, 2012-2013 


Use 


Wilcoxon Scores (Rank Sums) for Variable CRCT 
Classified by Variable Use 


N 


Sum of 
Scores 


Expected 
Under H 0 


SD Under 
Ho 


Mean Score 


1 349 125,994.50 122,673.50 2,686.03433 361.016759 

2 353 120,758.50 124,079.50 2,686.03433 342.092068 

Average scores were used for ties. 

Note: SD = Standard Deviation; H = Mull Hypothesis 

0 


Table 12 

Wilcoxon Two-Sample Test Results, 2012-2013 


Statistic 125994.50 

Normal 

Approximation 

z 1.2362 

One-Sided Pr > z 0.1082 

Two-Sided Pr > Izl 0.2164 


t Approximation 
One-Sided Pr > z 
Two-Sided Pr > Izl 


0.1084 

0.2168 


z includes a continuity correction of 0.5 
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Table 13 

Kruskal- Wallis Two-Sample Test Results for Non-Usage and Usage, 2012-2013 


Kruskal-Wallis Test 

Chi-Square 

1.5287 

DF 

1 

Pr > Chi-Square 

0.2163 


Figure 9 

Wilcoxon Box Plot Scores for CRCT by Non-Usage and Usage, 2012-2013 


Distribution of Wilcoxon Scores for CRCT 


Pr>Z 0.1GB2 
Pr>|Zl 0.2164 


Use 
































Figure 10 


Frequencies Above & Below the Median CRCT Scores by Non-Usage and Usage, 
2012-2013 


Frequencies Above and Below the Overall Median for CRCT 


Pr> Z 0.1165 
Pr >|Z| 02330 



1 2 
Use 


[ * Not Abov^ the Median 1 Above the Median 


The asymptotic p-value for the Kolmogorov-Smimov test is 0.07, which indicates a 
failure to reject the null hypothesis and that the CRCT score distributions are identical for the 
non-usage and usage groups (see Tables 14 and 15). The empirical distribution of CRCT scores 
shows the proportion of scores fall in the CRCT score ranges by non-usage, shown by the blue 
curve, and usage, shown by the red curve. The graph shows that the variation in distribution of 
CRCT scores across the non-usage and usage levels. Both curves for the non-usage and usage 
groups are closely aligned and follow similar trends with regard to the CRCT scores, and 
reiterating the fact that the non-usage or usage of Thinking Maps® is not able to explain 
sufficient variation in the CRCT scores to a statistically significant degree (see Figure 11). 
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Table 14 

Kolmogorov-Smirnov Asymptotic Results, 2012-2013 


KS 0.048868 D 0.097738 

KSa 1.294774 Pr>KSa 0.07 


Table 15 

Kolmogorov-Smirnov Empirical Distribution Test by Non-Usage and Usage, 
2012-2013 


Kolmogorov-Smirnov Test for Variable CRCT 
Classified by Variable Use 


Grade 

N 

EDF at Maximum 

Deviation from Mean 
at Maximum 

1 

349 

0.624642 

-0.918148 

2 

353 

0.72238 

0.912931 

Total 

702 

0.673789 



Maximum Deviation Occurred at Observation 72 
Value of CRCT at Maximum = 836.00 


Note. EDF = Empirical Distribution Function 



Figure 11 


Empirical Distribution Graph for CRCT Scores by Non-Usage and Usage, 2012- 
2013 


Empirical Distribution for CRCT 



Use - 1 - 2 


Friedman’s non-parametric ANOVA, which belongs to the PROC GLM library function 
in SAS, was used to determine the simultaneous influence of the sixth, seventh, and eighth 
grades and Thinking Maps® non-usage or usage on CRCT science scores. The model was 
significant (F = 23.30, p < .0001) (see Table 16). Grade level and Thinking Maps® together 
explained 9.11% variance in the CRCT scores (see Table 17). The interaction plot between grade 
levels marked on the X-axis and the Thinking Maps®, marked as non-usage by the blue line and 
usage by the red line, shows that the CRCT scores were higher for students who utilized 
Thinking Maps®. The seventh grade students had the highest CRCT scores for both the non¬ 
usage and usage groups (see Figure 12). It is difficult to determine whether the interaction 
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between the grade levels and Thinking Maps® is statistically significant because the CRCT 
scores follow a non-normal distribution and the scores that were sampled from the entire 
population of CRCT scores do not originate from the same distribution. 


Table 16 

Friedman’s Two-Way ANOVA Results, 2012-2013 

Friedman's Two-Way Non-Parametric ANOVA 
The GLM Procedure 
Dependent Variable: CRCT 


Source 

DF 

Sum of Scores 

Mean Square 

F Value 

Pr > F 

Model 

3 

76091.8734 

25363.9578 

23.30 

< .0001 

Error 

697 

758701.4190 

1088.5243 



Corrected 

Total 

700 

834793.2924 





Table 17 

R-Square Results, 2012-2013 

R-Square Coeff Var Root MSE CRCT Mean 


0.091151 


4.002739 32.99279 824.2553 



Figure 12 


Interaction Plot for CRCT Scores by Grade Level by Non-Usage and Usage, 2012- 
2013 


Interaction Plot for CRCT 



Descriptive Statistics, 2013-2014 

Sixth, seventh, and eighth grade students from a Title I middle school located in a 

suburban school district in middle Georgia provided the sample and focus of this study. Scores 
for 702 and 374 students for the 2012-2013 and 2013-2014 school years, respectively, were 
separated into two groups: (a) Students who did not use Thinking Maps® instruction in the 
classroom and (b) Students who did use Thinking Maps® instruction in the classroom. 
Appropriate statistical tests were conducted. 
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Table 18 provides information on descriptive statistics of CRCT scores based on usage or 
non-usage of Thinking Maps®. The 2013-2014 group data showed a total of 374 students, of 
which 182, or 48.67% of the total, were in the Non-Thinking Maps® or non-usage instructional 
group, and the Thinking Maps® or usage instructional group included 192 students, or 51.34% 
of the total number of tested students. In the non-usage group, the mean CRCT score was 820.90 
(n=182, SD=35.107), and the mean CRCT score for the usage group was 845.25 (m=192, 
SD=33.357) (see Table 18). The difference in means was 24.35, and the difference in standard 
deviations was 1.75. 

There were 72 sixth grade students comprising 51.06% of the population, who were in 
the Non-Thinking Maps® or non-usage instructional group, and 69 sixth grade students, or 
48.94%, who were in the Thinking Maps® or usage instructional group, for a total of 141 sixth 
grade students comprising 37.59% of the total population. The seventh grade comprised of 49 
students in the Non-Thinking Maps® or non-usage instructional group, or 46.67% of the 
population, and 56 students in the Thinking Maps® or usage instructional group, or 53.33% of 
the total number of students, totaling 105 students who were 28.07% of the total number of 
tested students. The eighth grade group numbered 61 in the Non-Thinking Maps® or non-usage 
instructional group, or 47.66%, and 67 in the Thinking Maps® or usage instructional group, or 
52.34% of the total number of eighth graders. This group comprised 34.22% of the total 
population of students taking the 2013-2014 science CRCT, or 128 students (see Table 19). 

Table 19 presents descriptive statistics of the CRCT science test scores by grade level. 

For the 2013-2014 CRCT science test administration, there was a total of 141 sixth grade 
students, comprising 37.59% of the population tested. The mean CRCT score for the sixth grade 
was 820.79 (m=141, SD=31.345). There was a total of 105 seventh grade students, making up 



28.07% of the population tested. The mean CRCT score for the seventh grade was 852.86 
(n=105, SD=39.038). There was a total of 128 eighth grade students, making up 34.22% of the 
population tested. The mean CRCT score for the eighth grade was 831.33 (n=128, SD=32.271). 
The difference between the means measured 32.07 between the sixth and seventh grades and 
21.53 between the seventh and eighth grades. The average difference between the means was 
26.80 (see Table 19). 

The confidence interval is defined as a range of the population that would occur in 95% 
of the cases on multiple occasions (NIST, 2013). The sixth grade results show a mean of 813.76 
(n=72, SD=34.486) for the non-usage group, and a mean of 828.12 (n=69, SD=25.966) for the 
usage group, the usage group showing a 14.36 point difference. The difference between the 
upper bound and lower bound scores in the 95% confidence interval for the sixth grade was 
16.21 for the non-usage group and 12.47 for the usage group, a difference of 3.74. The seventh 
grade non-usage group scored a mean of 850.04 (n=49, SD=34.40), and the usage group had a 
mean of 855.32 (n=56, SD=42.846). The mean scores for the non-usage and usage groups were 
closer in the seventh grade, but the non-usage group still scored higher than the usage group. 

The difference in the seventh grade confidence intervals was 19.76 for the non-usage group and 
22.95 for the usage group, a difference of 3.19. The non-usage group in the eighth grade scored 
a mean of 805.90 (n=61, SD=19.514), and the usage group scored a mean of 854.48 (n=67, 
SD=22.721). Contrary to the other two grades, the eighth graders who used Thinking Maps® 
scored an average of 48.58 points higher than the non-usage group. The difference in the eighth 
grade confidence intervals was 1.00 for the non-usage group and 11.08 for the usage group, a 
difference of 10.08. The mean CRCT scores for all of the Thinking Maps® non-usage groups 
scored lower than their usage counterparts (see Table 19). 
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Table 18 

Descriptive Statistics, 2013-2014 Science CRCT 



N 

Mean 

SD 

SE Mean 

Skewness 

SE 

Kurtosis 

SE 

Non-Usage 

182 

820.90 

35.107 

2.602 

0.345 

.180 

-0.494 

.358 

Usage 

192 

845.25 

33.357 

2.407 

0.734 

.175 

1.067 

.349 


Note. N = sample size; SID = standard deviation; St = standard error 


Table 19 

Statistics Non-Usage and Usage by Grade, 2013-2014 




6th Grade 



7th Grade 



8 th Grade 



Non- 

Usage 

Usage 

Total 

Non- 

Usage 

Usage 

Total 

Non- 

Usage 

Usage 

Total 

N 

72 

69 

141 

49 

56 

105 

61 

67 

128 

% Total 
Tested 

51.06 

48.94 

37.59 

46.67 

53.33 

28.07 

47.66 

52.34 

34.22 

Mean 

813.76 

828.12 

820.79 

850.04 

855.32 

852.86 

805.90 

854.48 

831.33 

SD 

34.486 

25.966 

31.345 

34.40 

42.846 

39.038 

19.514 

22.721 

32.271 

SE 

4.064 

3.126 

2.64 

4.914 

5.725 

3.81 

2.498 

2.776 

2.852 

Skewness 

0.288 

0.135 


-0.607 

0.672 


-0.269 

1.054 


SE 

0.283 

0.289 


0.34 

0.319 


0.306 

0.293 


Kurtosis 

-0.53 

-0.208 


-0.218 

-0.153 


0.348 

0.62 


SE 

0.559 

0.57 


0.668 

0.628 


0.604 

0.578 


95% Cl 










Upper Bound 

821.87 

834.35 


859.92 

866.80 


801.90 

860.02 


Lower Bound 

805.66 

821.88 


840.16 

843.85 


800.90 

848.94 



Non-Usage 


Usage Total 192 

Note. JN = sample size; SD = standard deviation; SD = standard error; Cl = confidence interval 



Tests of Normality, 2013-2014 

The histogram for the non-usage group showed a positive or right-skewed non-normal 
distribution with a skewness of .345 (SE = .180), while the histogram for the usage group 
showed a negative or left-skewed non-normal distribution with a skewness of .734 (SE = .175), a 
difference of .389 (see Figures 13 and 14). Westfall (2014) stated kurtosis “measures the 
departure from normality.” The kurtosis value of the non-usage group was -.494 (SE = .358), 
which had negative kurtosis and was platykurtic. This indicated the non-usage group varied from 
normal distribution. The kurtosis value of the usage group was 1.067 (SE = .349), which is said 
to have positive kurtosis, and was also platykurtic. This indicated the usage group varied from a 
normal distribution (see Table 20). Normality was not supported by the Shapiro-Wilk test for 
each of the two groups, non-usage and usage, because all p’s < .05 (see Table 20). Levene’s test 
for homogeneity of variances was used to determine that variances in the data set were equal 
when the data exhibited a non-normal distribution. Tests for homogeneity of variances showed 
F( 1,372) = 1.990, p = .159, which was not statistically significant because p > .05, indicating 
there are equal variances in the non-usage versus usage of Thinking Maps® in the sample 
population (see Table 21). 


Table 20 

Tests of Normality for Non-Usage and Usage, 2013-2014 


Kolmogorov-Smirnov a Shapiro-Wilk 



Statistic 

df 

Sig. 

Statistic 

df 

Sig- 

Non-Usage 

0.093 

182 

0.001 

0.975 

182 

0.002 

Usage 

0.110 

192 

0.000 

0.966 

192 

0.000 
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a. Lilliefors Significance Correction 


Table 21 

Levene's Test for Homogeneity of Variance, Non-Usage and Usage, 2013-2014 


Score 


Based on Mean 
Based on Median 
Based on Median 
and with adjusted df 
Based on trimmed 
mean 


Levene Statistic dfl 

1.990 1 

1.507 1 

1.507 1 

2.057 1 


d£2 

Sig, 

372 

0.159 

372 

0.220 

372.000 

0.220 

372 

0.152 


Figure 13 

Histogram of Science CRCT Scores, Non-Usage, 2013-2014 


Histogram 

for Usage_Nomisage= Nonusage 



Score 























Figure 14 

Histogram of Science CRCT Scores, Usage, 2013-2014 


Histogram 

for Usage_Nonusage= Usage 



The histogram for the sixth grade reported a mean of 820.79 (n=141, SD=31.345) and 
showed a skewness of .048 with a standard error of .204, and a kurtosis of -.402 with a standard 
error of .406 (see Figure 15). The histogram for the seventh grade reported a mean of 852.86 
(n=105, SD=39.038), and showed a skewness of .318 with a standard error of .236, and a 
kurtosis of .091 with a standard error of .467 (see Figure 16). The histogram for the eighth grade 
reported a mean of 831.33 (n=128, SD=32.271) and showed a skewness of .268 with a standard 
error of .214, and a kurtosis of .094 with a standard error of .425 (see Figure 17). All of them 
showed a non-normal distribution, and are all platykurtic. Normality was supported by the 
Shapiro-Wilk test for both sixth grade groups, and the two non-usage groups for the seventh and 
eighth grades because all p’s > .05, but normality was not supported by the Shapiro-Wilk test for 
both usage groups for the seventh and eighth grades because both p’s < .05 (see Table 22). 
Levene’s test for homogeneity of variances was used to determine that variances in the data set 
were equal when the data exhibited a non-normal distribution. Tests for homogeneity of 
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variances showed F(2,371) = 3.089, p = .047, which was statistically significant because p < .05, 
indicating there is not a difference between the variances grade level versus non-usage/usage of 
Thinking Maps® within the population (see Table 23). 


Figure 15 

Histogram of Sixth Grade CRCT Scores, 2013-2014 


Histogram 
for Grade= 6th grade 






















Figure 16 

Histogram of Seventh Grade CRCT Scores, 2013-2014 


Histogram 
for Grade= 7th grade 



Figure 17 

Histogram of Eighth Grade CRCT Scores, 2013-2014 


Histogram 
for Grade= 8th grade 



Mean = 831.33 
Std. Dev. = 32.271 
N = 128 


Score 
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Table 22 

Tests of Normality for Sixth, Seventh, and Eighth Grade CRCT Scores, 2013-2014 


Kol mogorov-S m i rno v a Shapiro-Wilk 


Non-Usage 


Level 

Usage 

Statistic 

df 

Sig- 

Statistic 

df 

Sig. 

6 

Non-Usage 

.081 

72 

.200 

.974 

72 

.146 


Usage 

.066 

69 

.200 

.990 

69 

.860 

7 

Non-Usage 

.104 

49 

.200 

.958 

49 

.076 


Usage 

.099 

56 

.200 

.950 

56 

.022 

8 

Non-Usage 

.092 

61 

.200 

.977 

61 

.310 


Usage 

.175 

67 

.000 

.898 

67 

.000 


IT Lillietors Significance Correction 


Table 23 

Levene’s Test for Homogeneity of Variance, Grades 6, 7, and 8, 2013-2014 


Score Based on Mean 
Based on Median 
Based on Median 
and with adjusted df 
Based on trimmed 
mean 


Levene Statistic dfl 

3.089 2 

3.107 2 

3.107 2 

3.107 2 


df2 

Sig, 

371 

0.047 

371 

0.046 

357.450 

0.046 

371 

0.046 


Table 24 

Tukey & LSD Post hoc Comparisons by Grade Level, 2013-2014 


95% Cl 





Mean Difference 

SE 

Sig- 

Lower 

Upper 

6 

7 

-32.07 

4.38 

0.000 

-42.38 

-21.76 


8 

-10.541 

4.148 

0.031 

-20.30 

-0.78 

7 

6 

32.07 

4.38 

0.000 

21.76 

42.38 


8 

21.529 

4.474 

0.000 

11.00 

32.06 

8 

6 

10.541 

4.148 

0.031 

0.78 

20.30 

7 

-21.529 

4.474 

0.000 

-32.06 

- 11.00 

6 

7 

-32.07 

4.38 

0.000 

-40.68 

-23.46 


8 

-10.541 

4.148 

0.011 

-18.70 

-2.38 

7 

6 

32.07 

4.38 

0.000 

23.46 

40.68 


8 

21.259 

4.474 

0.000 

12.73 

30.33 

8 

6 

10.541 

4.148 

0.011 

2.38 

18.70 

7 

-21.529 

4.474 

0.000 

-30.33 

-12.73 


Note. Cl = Confidence Interval; 


Transformations, 2013-2014 

The Shapiro-Wilk and Kolmogorov-Smimov normality tests indicated the scores for the 
three grade levels were significant for both groups, so there was no need to transform the scores 
to reach a normal distribution (see Table 22). Levene’s test for homogeneity of variance showed 
F( 2, 371) = 3.089, p = .047, which was statistically significant because p < .05 (see Table 23). 
The p value, while less than .05, would round to .05, so it was reaching normal distribution and 
variances between the grade levels. Post hoc comparisons using the Tukey HSD test indicated 
that there was a statistical difference between the eighth and sixth grade scores and the eighth 
and seventh grade scores, but there was very little difference between the seventh and eighth 


grade means (see Table 24). The LSD test confirmed the significance of the scores on the Tukey 
HSD test (all p’s < .05) (see Table 24). The grade level had a statistically significant effect on 
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the achievement scores because r| = 0.126. This small effect size showed that 12.60% of the 
variance in CRCT scores is attributed to grade level. 

The non-usage and usage groups did not follow a normal distribution and it was 
necessary to perform a log transformation and complete a Mann-Whitney test to determine the 
extent of the differences in variance. 


Non-Parametric Analysis, 2013-2014 

All non-parametric statistical analyses were conducted in SAS. The PROC NPAR1WAY 

is a procedure in the SAS library that is built-in to perform non-parametric analyses. The CRCT 
science scores followed a normal distribution. The ANOVA for each of the grade levels of sixth, 
seventh, and eighth grades showed a mean score of 820.79, 852.86, and 831.33, respectively (see 
Table 25). The model was significant (F = 27.1691, p < .0001), indicating that grade level has an 
effect on the CRCT scores (see Table 26). The box plot depicts the seventh grade as having 
higher mean scores than the sixth and eighth grades (see Figure 18). 


Table 25 

ANOVA Results for CRCT Scores by Grade Level, 2013-2014 


Analysis of Variance for Variable CRCT 


Classified by Variable Grade 


Grade 

N 

Mean 

8 

141 

820.787234 

7 

105 

852.857143 

6 

128 

831.328125 



Table 26 


ANOVA Results F-Statistic for Grade Level, 2013-2014 


Source 

DP 

Sum of Squares 

Mean Square 

F Value 

Pr > F 

Among 

2 

62,730.94612 

31,365.47306 

27.1691 

<.0001 

Within 

371 

428,302.6929 

IJ54.4547 




Average scores were used for ties. 


Figure 18 

Distribution ofCRCT Scores by Grade Level, 2013-2014 


Distribution of CRCT 
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The Kruskal-Wallis test was significant (/2 = 42.7762, p < .01) (see Table 27). Grade 
level had an effect on the CRCT scores to a statistically significant degree. There is enough 
evidence to reject the claim that CRCT-ranked mean scores from the three grade levels have 
equally ranked mean scores. The Wilcoxon rank sum test also shows the same picture as the 
Kruskal-Wallis test where not only the sum of scores was above the standard deviation of Ho, but 
also ranked mean score is different across grade levels with seventh grade having the highest 
CRCT-ranked mean score of 241.69. The CRCT mean scores, as well as the ranked mean scores 
are influenced most by the seventh grade, followed by the eighth grade and then the sixth grade 
(see Table 28). The distribution of CRCT-ranked sum scores in the box plot graphically shows 
differences in the quartile scores based on grade levels (see Figure 19). The median scores 
above the mean showed the seventh grade had the highest number of scores above the median, 
with a mean of .705, followed by the eighth grade with .50 and then the sixth grade with .34 (see 
Table 29). The median one-way analysis was significant (y = 31.2131, p < .01) (see Table 30). 
While the frequency graph shows a greater frequency in the sixth and eighth grade scores, there 
were a larger number of scores that were above the median for the seventh grade than for the 
other two, followed by the eighth grade and then the sixth grade (see Figure 20). 


Table 27 

Kruskal-Wallis Chi-Square Results by Grade Level, 2013-2014 


Chi-Square 

DF 

Pr > Chi-Square 


42.7762 

2 

<.0001 



Table 28 


Wilcoxon Test Results by Grade Level, 2013-2014 


Wilcoxon Scores (Rank Sums) for Variable CRCT 




Classified by Variable Grade 


Grade 

N 

Sum of 
Scores 

Expected 
Under H 0 

Std Dev 
Under H 0 

Mean Score 

8 

141 

21,272.00 

26,437.50 

1,013.01999 

150.865248 

7 

105 

25,377.50 

19,687.50 

939.29344 

241.690476 

6 

128 

23,475.50 

24,000.00 

991.75189 

183.402344 


Average scores were used for ties. 



Figure 19 

Distribution ofWilcoxon Scores by Grade Level, 2013-2014 
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Distribution ofWilcoxon Scores for CRCT 



Table 29 

Median Scores for CRCT Scores by Grade Level, 2013-2014 


Median Scores (Number of Points Above Median) for Variable 
CRCT 


Grade N 


Classified by Variable Grade 

Sum of Expected Std Dev 
Scores Under H 0 Under H 0 


Mean Score 


8 141 49.00 70.50 4.64927 0.347518 

7 105 74.00 52.50 4.31090 0.704762 







































6 128 64.00 64.00 4.55166 

Average scores were used for ties. 


Table 30 

Median One-Way Analysis Chi-Square Results, 2013-2014 


Median One-Way Analysis 

Chi-Square 

31.2131 

DF 

2 

Pr > Chi-Square 

0.0001 


0.500000 
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Figure 20 

Frequency Graph for CRCT Scores Above and Below the Overall Median, 2013- 
2014 


Frequencies Above and Below the Overall Median for CRCT 



~i-1-r 

6 7 8 

Grade 


■ Not Above the Median ■ Above the Median 


The asymptotic p-value for the Kolmogorov-Smimov test is 2.909181, which indicates a 
rejection of the null hypothesis that CRCT score distributions are identical for the three grade 
levels, and the lowest empirical deviation from the mean at 0.309417 (see Tables 31 and 32). 
The empirical distribution of CRCT scores shows the proportion of scores falling within the 
CRCT score ranges. The graph shows that the variation in the distribution of CRCT scores is 
different across the three grade levels, but the magnitude of difference is more in the seventh 
grade scores when compared to the sixth and eighth grades. The proportion of CRCT scores 
















falling within a range is marked on the Y-axis, and the CRCT score ranges are marked on the X- 
axis. The graph shows more variation in the seventh grade curve of the CRCT score distribution, 
especially after the 825 mark, as indicated by a steeper curve when compared to the sixth and 
eighth grade curves (see Figure 21). 


Table 31 

Kolmogorov-Smirnov Test Results by Grade Level, 2013-2014 


Kolmogorov-Smirnov Test for Variable CRCT 
Classified by Variable Grade 


Grade 

N 

EDF at 

Maximum 

Deviation from 
Mean at Maximum 

6 

141 

0.773050 

1.369060 

7 

105 

0.419048 

-2.446013 

8 

128 

0.726563 

0.778479 

Total 

374 

0.657754 



Maximum Deviation Occurred at Observation 282 
Value of CRCT at Maximum = 845.00 


Table 32 

Kolmogorov-Smirnov Asymptotic Results, 2013-2014 


KS 0.150430 KSa 2.909181 



Figure 21 

Empirical Distribution for CRCT Science Scores, 2013-2014 
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Empirical Distribution for CRCT 



The Wilcoxon two-sample test was used to determine if there was an influence of non¬ 
usage or usage of Thinking Maps® on the variation of CRCT science scores. The mean CRCT 
score for the non-usage and usage groups was 820.90 and 845.25, respectively. The Wilcoxon 
mean-ranked CRCT scores for non-usage (coded as 1) was 150.59, and for usage (coded as 2) 
was 223.66 (see Table 33). 

The Wilcoxon two-sample test suggests there is a statistically significant difference, 
indicating that Thinking Maps® use does have an influence on the variation in CRCT scores (see 






















Tables 34 and 35). There is sufficient evidence to reject the claim that CRCT-ranked mean 
scores for the non-usage and usage groups have equal ranked-mean scores. The distribution of 
CRCT-ranked sum scores in the box plot graphically represents differences in the quartile scores 
based on non-usage and usage levels (see Figure 22). The box plot also reveals the same picture 
where the distribution for non-usage and usage are similar (see Figure 23). The bar graph 
depicts that there is an approximately equal distribution of median CRCT scores above and 
below the mean for both non-usage and usage groups, but there is a much larger number of 
scores above the mean for the usage group than for the non-usage group (see Figure 24). 


Table 33 

Wilcoxon Test Scores for Non-Usage and Usage, 2013-2014 


Use 


Wilcoxon Scores (Rank Sums) for Variable CRCT 
Classified by Variable Use 


N 


Sum of 
Scores 


Expected 
Under H 0 


Std Dev 
Under H 0 


Mean Score 


1 183 27557.00 34404.00 1049.02449 150.584699 

2 192 42943.00 36096.00 1049.02449 223.661458 

Average scores were used for ties. 


Table 34 

Wilcoxon Test Results, Non-Usage and Usage, 2013-2014 


Wilcoxon Two-Sample Test 


Statistic 

Normal Approximation 


27557.00 
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z -6.5265 

One-Sided Pr > z < .001 

Two-Sided Pr > Izl < .001 

t Approximation 

One-Sided Pr > z < .001 

Two-Sided Pr > Izl < .001 


z includes a continuity correction of 0.5 


Table 35 

Kruskal-Wallis Test Results, Non-Usage and Usage, 2013-2014 


Chi-Square 42.6019 

DF 1 

Pr > Chi-Square < .0001 



Figure 22 

Wilcoxon Test Results Box Plot, 2013-2014 


Distribution of Wilcoxon Scores for CRCT 
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Figure 23 

Frequency Graph for CRCT Scores Above and Below the Overall Median, 2013- 
2014 
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Frequencies Above and Below the Overall Median for CRCT 
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Figure 24 

Empirical Distribution Graph for CRCTScores, Non-Usage and Usage, 2013-2014 


Empirical Distribution for CRCT 



Use - 1 - 2 


Friedman’s non-parametric ANOVA, which belongs to the PROC GLM library function 
in SAS was used to determine the simultaneous influence of grade level (sixth, seventh, and 
eighth) and Thinking Maps® (non-usage and usage) on the CRCT scores. The model was 
significant because F = 37.44, p < .0001 (see Table 36). Grade level and Thinking Maps® 
together explained approximately 23.2% of the variance in the CRCT scores (see Table 37). The 
interaction plot between grade levels, marked on the X-axis, and Thinking Maps®, with non- 
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usage shown by the blue line and usage marked by the red line, shows that the CRCT scores 
were higher for students who utilized Thinking Maps® (see Figure 25). The seventh grade 
students had the highest CRCT scores for both non-usage and usage groups and this trend was 
prevalent across all grade levels. It is difficult to determine whether the interaction between 
grade levels and Thinking Maps® is statistically significant because the CRCT scores follow a 
non-normal distribution, and the scores that were sampled from the entire population of CRCT 
scores did not originate from the same distribution. 


Table 36 

Friedman’s Two-Way Non-Parametric ANOVA Test Results, 2013-2014 
Friedman's Two-Way Non-Parametric ANOVA 


The GLM Procedure 


Source 


DF 


Dependent Variable: CRCT 


Sum of Scores 


Mean 

Square 


F Value 


Model 3 114,358.4299 38,119.4766 37.44 


Error 370 376,675.2092 1,018.0411 

Corrected 

Total 373 491,033.6390 


Pr > F 


< .0001 


Table 37 

R-Squared Results, 2013-2014 


R-Square 

Coeff Var 

Root MSE 

CRCT Mean 

0.232893 

3.828512 

31.90676 

833.3984 



Figure 25 


Interaction Plot for CRCT Scores by Grade Level for Non-Usage and Usage, 2013- 
2014 


Interaction Plot for CRCT 



Research Questions 

The purpose of this study was to determine if the use of Thinking Maps® had an effect 
on the achievement of middle grades students in the science classroom by an analysis of scores 
from the 2012-2013 and 2013-2014 administrations of the Georgia CRCT. In addition, science 
teachers’ perceptions of the usage or non-usage of Thinking Maps® was investigated to 
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determine if a relationship existed between the teachers’ perceptions and the implementation of 
Thinking Maps® in middle grades science. 

Research Question 1: To what extent do Thinking Maps® affect middle grades students’ 
academic achievement in science? 

Research Question 2: What are teachers’ perceptions about the use of Thinking Maps® 
with middle grades students? 

Research Question 3: What, if any, is the relationship between the use of Thinking 
Maps® and teachers’ perception and implementation in middle grades? 

Data Analysis 

All statistical tests such as the Kolmogorov-Smimov, Kruskal-Wallis, Wilcoxon rank 
sum scores, and Friedman’s analysis were performed using SPSS version 24, the Statistical 
Package for the Social Sciences®, published by IBM, and the SAS Institute’s Statistics Analysis 
Software (SAS/STAT®) statistical analysis software packages. For this study, the non-usage or 
usage of Thinking Maps® in the science classroom served as the independent variables. The 
dependent variable was the student achievement in science for grades six through eight as 
measured on the 2012-2013 and 2013-2014 administrations of the Georgia Criterion-Referenced 
Competency Tests (CRCT). 

The 2012-2013 CRCT administration showed a total of 702 students, with a mean of 
826.20 and a standard deviation of 34.851. The Non-Thinking Maps® or non-usage 
instructional group had a total of 349 students in it. The mean science test score for the Non- 
Thinking Maps® or non-usage instructional group was 825.94, with a standard deviation of 
36.50. The Thinking Maps® or usage instructional group included 353 students in it, who 



scored a mean of 826.45, with a standard deviation of 33.202. In comparing the means of both 
groups, there was only a 0.51 difference. 

Non-parametric statistical tests were conducted for the data following a non-normal 
distribution. The Kruskal-Wallis test was significant, showing that grade level had an effect on 
the CRCT scores to a statistically significant degree (see Table 8). The Wilcoxon-ranked sum 
scores test is a non-parametric form of the t-test, and ranked CRCT scores by each grade level. 
For the 2012-2013 data, the Wilcoxon test showed the seventh grade having the highest-ranked 
mean score of 437.16 (see Table 7). This was confirmed by the box plot and the frequency bar 
graph (see Figures 6 and 7). Additionally, the empirical distribution graph for CRCT scores by 
grade level shows that the seventh grade had an effect on CRCT scores, with a greater number of 
scores occurring above the 850 score (see Figure 8). 

However, the Rruskal-Wallis test was not significant for non-usage and usage, indicating 
that usage of Thinking Maps® did not have an influence on the CRCT scores to a statistically 
significant degree (see Table 13). The Wilcoxon test was used to show the influence of non¬ 
usage and usage of Thinking Maps® on CRCT science scores. The Wilcoxon was not 
significant for the 2012-2013 data, meaning Thinking Maps® did not influence the variation in 
CRCT scores. Because the means were close, there was not sufficient evidence to reject the 
claim that non-usage and usage had an effect on scores, as shown in Table 11. This was shown 
on the box plot and bar graph as well (see Figures 9 and 10). Additionally, the empirical 
distribution graph for non-usage and usage showed the two curves as closely aligned (see Figure 
11 ). 

Friedman’s non-parametric ANOVA was used to determine the influence of both grade 


level and amount of usage on the CRCT science scores. The model was significant, showing that 
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grade level and usage of Thinking Maps® together explained 9.11% of the variance in the CRCT 
scores (see Tables 16 and 17). Figure 17 shows the interaction between grade levels and non¬ 
usage or usage, and shows the CRCT scores for those students who used Thinking Maps® were 
higher. However, it is difficult to determine if the interaction between grade levels and Thinking 
Maps® were significant because the CRCT scores followed a non-normal distribution, and the 
CRCT scores did not originate from the same distribution. 

The 2013-2014 CRCT administration showed a total of 374 students, with a mean of 
833.075 and a standard deviation of 34.232. The Non-Thinking Maps® or non-usage group had 
a total of 182 students in it. The mean science test score for the Non-Thinking Maps® or non¬ 
usage group was 820.90, with a standard deviation of 35.107. The Thinking Maps® or usage 
instructional group included 192 students, who scored a mean of 845.25, with a standard 
deviation of 33.357. In comparing the means of both groups, there was a difference of 24.35. 

Non-parametric statistical tests were conducted for the data following a non-normal 
distribution. The Kruskal-Wallis test was significant, showing that grade level had an influence 
on the CRCT scores to a statistically significant degree (see Table 27). The Wilcoxon rank sum 
test ranked the CRCT scores by grade level. For the 2013-2014 data, the Wilcoxon test showed 
that the seventh grade influenced the CRCT scores the most, followed by the eighth, and then the 
sixth grades (see Table 28). The box plot and frequency bar graph show a greater number of 
CRCT scores were above the mean for the seventh grade than in the sixth grade or eighth grade 
(see Figures 19 and 20). The empirical distribution graph for the three grade levels indicated a 
greater number of scores in the seventh grade as having a higher mean than the sixth and eighth 
grades, notably after the 825 score (see Figure 21). 



The Kruskal-Wallis test for non-usage and usage was significant (see Table 35). It shows 
that usage of Thinking Maps® had an influence on the CRCT scores to a statistically significant 
degree. The Wilcoxon two-sample test was used to determine if there was an influence of non¬ 
usage or usage of Thinking Maps® on the CRCT science scores. It suggests there is a 
statistically significant difference, indicating that Thinking Maps® do have an influence on 
CRCT scores (see Tables 34 and 35). The box plot confirms that the distributions for non-usage 
and usage are similar (see Figure 22). The bar graph graphically depicts that there were a larger 
number of scores above the mean for the usage group than for the non-usage group (see Figure 
23). The empirical distribution graph for non-usage and usage indicates a greater difference of 
means in the usage versus the non-usage group, especially between the 800-850 score range (see 
Figure 24). 

Friedman’s non-parametric ANOVA was used to determine if there was an influence 
between the three grade levels and Thinking Maps® non-usage or usage in terms of CRCT 
science scores. The test was significant (see Table 36). Grade level and usage together 
explained 23.2% of the variance in the CRCT scores (see Table 37). The interaction plot shows 
that the CRCT scores were higher for students who used Thinking Maps® in their science class 
(see Figure 25). The seventh grade had higher scores than the other two grade levels. 

It is difficult to determine the interaction between grade levels and Thinking Maps® is 
statistically significant because of the non-normal distribution of the CRCT scores and the scores 
that were sampled from the population of CRCT scores did not originate from the same 
distribution. Because an interaction effect is difficult to interpret with non-normal data, to show 
a relationship, there must include a comparison of the means for each grade level with respect to 
the non-usage and usage as reported in the descriptive results (see Table 38). 
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Table 38 

Comparison of Means by Grade and Usage, CRCT Science Scores 


Comparison of Means 

2012-2013 2013-2014 


Grade 

Level 

Non-Usage 

Mean 

Usage 

Mean 

Non-Usage 

Mean 

Usage 

Mean 

6 

840.13 

802.66 

813.76 

828.12 

7 

840.65 

837.46 

850.04 

855.32 

8 

795.00 

831.42 

805.90 

854.48 


Teacher Perception Survey Results 

Each science teacher and special education science co-teacher was asked to complete a 
Thinking Maps® Teacher Perception Survey developed by the researcher (see Appendix A). 
Thirteen surveys were completed and returned, six from regular education teachers and seven 
from the special education science co-teachers, one of which teaches gifted. 

Demographic data analysis showed 61.54% of the respondents were above the age of 41, 
84.62% who responded were female, 84.62% were white, 15.38% were black, and the mean 
amount of teaching experience was 16 years. 

The selected school received initial training in Thinking Maps® during the 2008-2009 
school year. Of the respondents, 11 of the 13 science teachers, or 84.62%, received the training 
during the first year of Thinking Maps® implementation. Of those who did not receive the 
training in the first year, one of the special education co-teachers received training in another 


state before moving to the selected school. The other teacher received informal training from a 



member of the original team of educators from the selected school who was trained as a 
Thinking Maps® trainer for the school system. 

The Teacher Perception Survey asked teachers if they believed Thinking Maps® help 
students learn science content. Of the respondents, 84.62%, or 11 of the 13, believe Thinking 
Maps® help students learn science content. While this does look like a high percentage, several 
teachers discussed either a reluctance to use Thinking Maps® or challenges inherent to using 
Thinking Maps®. For example. Teacher 7.1, who did not use Thinking Maps®, said, “I think it 
took things we already used and put a new name on it,” and then indicated that they were “not 
effective.” This does not mean this teacher did not use them as directed by the school system, 
however, and as indicated on the survey, the double bubble map, flow map, and tree map were 
the ones used most in this teacher’s classes and the ones the teacher felt were most effective. 
Teacher 8.1, who did not feel Thinking Maps® were effective also indicated very little use 
Thinking Maps® in the classroom, felt that “Thinking Maps® forces [sic] students to process 
information in a way that may not be natural to them.” This teacher went on to say, “I don’t 
organize information that way so it’s hard for me to see the opportunities to use them.” Teacher 
6.1 did not say if she felt they were effective, but she professed a reluctance to use them because, 
“They don’t like or think they need the maps.” Table 39 shows a comparison of mean CRCT 
scores for the teachers who did not utilize Thinking Maps® and those who utilized Thinking 
Maps® in their instructional practice. 


Table 39 


Comparison of CRCT Means by Teacher, Grade, and Non-Usage or Usage 


Mean CRCT 
Score 


Did Not Use Thinking Maps 


Used Thinking Maps 


2012-2013 


2013-2014 


2012-2013 


2013-2014 



Ill 


Teacher 1 Teacher 2 


6 

840.13 

813.76 

802.66 

828.12 

7 

840.65 

850.04 

837.46 

855.32 

8 

795.00 

805.90 

831.42 

854.48 


Note: Teachers in narrative = 6.1 (non-usage) & 6.2 (usage); 7.1 (non-usage) & 7.2 (usage); 8.1 (non-usage) &. 8.2 (usage) 


Teacher 8.2 discussed challenges when using Thinking Maps® in the classroom. “It was 
challenging, at times, to encourage students to expand their thinking [sic] to add more than just 
facts.” One challenge teachers discussed across grade levels was how to incorporate different 
maps, and multiple maps, into their practice. Teacher 6.2 said, “The most challenging aspect has 
been trying to use more than the two that I use the most. Trying to use a variety of maps is 
difficult for me to incorporate.” Teacher 7.2 said, “[The most challenging aspect is] finding one 
to use for every standard/element.” The gifted science teacher wrote, “[The most challenging 
aspect is] using a variety of Thinking Maps®. Some maps lend themselves more easily to 
science concepts [sic] some maps are less useful than others.” Besides these challenges, another 
challenge discussed was time. One special education co-teacher professed, “The most 
challenging aspect.. .is having the time during instruction for their use. There are so many 
standards that must be covered that students don’t have adequate time to develop understanding 
or [sic] able to organize details on the Thinking Maps® before the curriculum/instruction moves 
forward.” Another co-teacher agreed. “I can see the value in creating them, but unfortunately, 
we just don’t have the time to take two or three days to develop them the way they probably 
should be.” 

The Teacher Perception Survey also asked about which three Thinking Maps® science 


teachers felt were most effective, which three they felt were least effective, and which three they 



used the most. Teachers indicated they felt the tree map, flow map, and double bubble map were 


the most effective, in that order. Conversely, those maps the teachers felt least effective were the 
bubble map, circle map, and the bridge map, followed closely by the brace map. The Thinking 
Maps® used the most were the tree map, the flow map, and the double bubble map, followed by 
the brace map (see Table 40). 

Lastly, the Teacher Perception Survey asked science teachers to indicate how often they 
used Thinking Maps® each week on average and each day with their classes. The science 
teachers’ responses were that 53.85% used them once per week and 38% used them four to five 
times per day, an indication they used them with each of their content-area classes. This 
indicates usage in honors or accelerated classes as well as in interrelated or co-taught classes 
with a special education teacher. 


Table 40 


Data from Thinking Maps® Teacher Perception Survey 


Thinking Man 

Most Effective 

Least Effective 

Used Most 

Primary Purpose 

Brace 

3 

6 

5 

Whole-to-Part Relationship 

Bridge 

2 

7 

1 

Analogy 

Bubble 

1 

10 

1 

Describing using adjectives 

Circle 

0 

7 

0 

Define in context; Brainstorming 

Double Bubble 

9 

2 

11 

Comparison/Contrast 

Flow 

12 

1 

11 

Sequence of Events 

Multi-Flow 

2 

2 

0 

Cause/Effect Relationships 

Tree 

13 

0 

13 

Classifying; Grouping 
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Summary 

The results of the data analysis collected from the scores from the 2012-2013 and 2013- 
2014 administrations of the CRCT are presented in Chapter 4 with analyses. The CRCT scores 
were initially analyzed through SPSS, but because the scores followed a non-normal distribution, 
it was necessary to perform non-parametric tests using SAS. Tests were completed based on the 
three grade levels (sixth, seventh, and eighth), non-usage or usage of Thinking Maps® in 
instructional practice, and to determine interaction between the grade level and non-usage or 
usage. 

The 2012-2013 Kruskal-Wallis test was statistically significant for the sixth, seventh, and 
eighth grades. The Wilcoxon rank sum test gave a similar picture as the Kruskal-Wallis, 
whereby the seventh grade influenced the most variance in the CRCT scores when compared to 
the sixth grade and eighth grade. The box plot showed the differences in the quartiles based on 
grade level. The frequency bar graph also showed that the seventh grade had a higher frequency 
of CRCT scores than the other two grade levels. The Kolmogorov-Smirnov asymptotic p-value 
indicated a rejection of the null hypothesis for the three grade levels. The empirical distribution 
graph showed that the magnitude of the distribution was greater for the seventh grade, especially 
after the 850 score. 

The Kruskal-Wallis test for non-usage and usage of Thinking Maps® was not significant, 
and the Wilcoxon test for non-usage and usage was not significant, either. This indicated that 
Thinking Maps® usage did not influence the variation in CRCT scores. The box plot did not 
depict large differences based on non-usage or usage, and there was a similar distribution. The 
bar graph also showed an approximately equal distribution of the median CRCT scores above 
and below the median for both non-usage and usage groups. 



Friedman’s non-parametric ANOVA was used to determine the simultaneous influence of 
grade level and non-usage or usage of Thinking Maps® on CRCT scores. The Friedman’s test 
was significant, together explaining that 9.11% of the variance in CRCT scores were influenced 
by grade level and usage of Thinking Maps®. The interaction plot showed that CRCT scores 
were higher for the students who used Thinking Maps®. The seventh grade has the highest 
scores. 

The 2013-2014 ANOVA was significant for the sixth, seventh, and eighth grades, 
indicating that grade level had an effect on the CRCT scores. The box plot revealed that seventh 
grade had a higher mean score than the sixth or eighth grades. 

The Kruskal-Wallis test was significant. Grade level had an effect on the CRCT scores to 
a statistically significant degree. The Wilcoxon test depicted a similar picture, with the seventh 
grade having the highest ranked mean score. The box plot showed differences in the quartiles 
based on the grade levels. The seventh grade also had the highest number of scores above the 
median, then eighth grade, followed by sixth grade. The frequency graph showed a greater 
frequency in the sixth and eighth grade scores, but the larger number of scores that were above 
the median were from the eighth grade, but the larger number of scores that were above the 
median were from the seventh grade. The Kolmogorov-Smimov asymptotic /;-value indicated a 
rejection of the null hypothesis for the three grade levels. The empirical distribution graph 
showed that a variation in the distribution of CRCT scores was different across the three grade 
levels, but the magnitude of the difference is more in the seventh grade scores than the sixth and 
eighth grades. The variation was shown by a steeper curve on the seventh grade curve when 
compared to the sixth and eighth grade curves. 
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The Kruskal-Wallis and Wilcoxon tests were both significant for non-usage versus usage, 
suggesting there is a statistically significant difference, that Thinking Maps® usage did have an 
influence on the variation in CRCT scores. There is sufficient evidence to reject the claim that 
ranked-mean scores for non-usage and usage have equal-ranked mean scores. The box plot 
shows differences in quartiles based on non-usage and usage. The bar graph showed an 
approximately equal distribution of median CRCT scores above or below the mean, but there 
were more scores above the mean for the usage group than for the non-usage group. The 
empirical distribution graph showed more scores between the 800-850 range for the usage group 
than for the non-usage group. 

Friedman’s ANOVA was used to determine the simultaneous influence of grade level and 
usage on the CRCT scores. The test was significant, explaining that 23.2% of the variance in the 
CRCT scores could be attributed to both grade level and usage of Thinking Maps®. The seventh 
graders had the highest scores in the usage group. 

Most teachers (67%) who reported on the Teacher Perception Survey that they had a 
positive perception of and used Thinking Maps® regularly and consistently in their classes had a 
higher mean of scores in their classes than those who had a negative perception of or did not use 
Thinking Maps® regularly in their classes. 

Based on the data results from the non-parametric testing and perceptions of teachers 
who felt favorably toward using Thinking Maps® regularly in their science classes, it can be 
concluded there is a trend in the favorable perception and consistent usage of Thinking Map® in 
the middle grades science classes and the increase in achievement score means of the students 


who use them regularly. 



Because the interaction effect cannot be calculated due to the variance in the data sets to 
report a relationship, a comparison of the means for each grade level with respect to the non¬ 
usage and usage as reported in the descriptive results is found in Table 39. 


CHAPTER 5 

FINDINGS, DISCUSSION, CONCLUSIONS 

The purpose of this study was to determine if there was a relationship between the 
achievement scores on the Georgia Criterion-Referenced Competency Test (CRCT) of students 
who used Thinking Maps® in their middle school science classes to leam content information as 
compared to students who did not use Thinking Maps®. The study focused on a statistical 
analysis of the scores for classes where Thinking Maps® were used and classes where Thinking 
Maps® were not used. Additionally, the perceptions of the science teachers in terms of the non¬ 
usage or usage of Thinking Maps® were analyzed. The final analysis was to determine if there 
was a relationship between the perceptions of the teachers and the non-usage or usage of the 
Thinking Maps® as related to achievement scores on the science assessments over two years. 

All participants in the study were in one middle Georgia Title I middle school having grades six, 
seven, and eight. 

This chapter presents the research questions and the findings based on the review of 
literature, data analysis of the Georgia CRCT science scores for 2012-2013 and 2013-2014, and 
teacher perception surveys. Conclusions will be drawn based on the results of the data analysis 


and compared to the findings in the review of literature. 
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SUMMARY 

The focus of the study was on achievement scores in middle grades science over two 
years. The dependent variable was the achievement score of 702 students on the Georgia CRCT 
in science over the 2012-2013 school year and 374 students over the 2013-2014 school year 
across the three grade levels (i.e., 6 th , 7 th , and 8 th grades). The independent variables were the 
non-usage and usage of Thinking Maps® in the science classes across the three grade levels. 

The population consisted of 702 middle school students in the 2012-2013 school year, and 374 
middle school students in the 2013-2014 school year. Science teachers, special education 
collaborative science teachers, and one gifted education collaborative science teacher were 
surveyed to ascertain their perception of Thinking Maps®, and if, when, how, and which maps 
they used in the instructional setting. The study focused on three research questions. 

Research Question 1 

To what extend do Thinking Maps® affect middle grades’ student’s academic 
achievement in science? 

Non-parametric versions of statistical tests were performed in SPSS and SAS based on 
non-normal distributions using transformed data on both the 2012-2013 and 2013-2014 Georgia 
CRCT science scores. Tests were completed for the sixth, seventh, and eighth grade levels, the 
non-usage or usage of Thinking Maps®, and to determine interaction between the grade levels 
and amount of usage. 

For the 2012-2013 data, Kruskal-Wallis, Wilcoxon, and Kolmogorov-Smimov statistical 
tests showed that the greatest influence on CRCT science scores was in the seventh grade. The 
Kruskal-Wallis and Wilcoxon tests for non-usage versus usage of Thinking Maps® showed that 
usage of Thinking Maps® did not influence the variation in the CRCT scores. Friedman’s non- 



parametric ANOVA was performed to determine the simultaneous influence of grade level and 
non-usage or usage of Thinking Maps®, and showed that 9.11% of the variance was influenced 
by grade level and usage. 

The 2013-2014 statistical tests indicated that grade level had an effect on the CRCT 
scores, and again, the greatest influence on CRCT science scores was in the seventh grade. The 
non-usage versus usage tests showed that usage of Thinking Maps® did have an influence on the 
variation of CRCT scores. Friedman’s non-parametric ANOVA showed that 23.2% of the 
variance in the CRCT scores could be attributed to both grade level and usage of Thinking 
Maps®. 

Research Question 2 

What are teachers’ perceptions about the use of Thinking Maps® with middle grades 
students? 

Thirteen science and special education teachers responded to the Thinking Maps® 
Teacher Perception Survey developed by the researcher (see Appendix). Six were the regular 
education science teachers at the selected school, six were special education co-teachers, and one 
gifted education teacher. 

Of the thirteen respondents, 84.62% said they believed Thinking Maps® help students 
learn science content. However, several teachers revealed they did not use them regularly 
because they either did not like using them or felt they were effective, or because of the 
challenges of incorporating them into existing curriculum. Challenges such as finding one for 
each element in a unit or trying to incorporate a variety of maps into the instruction, as well as 
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having the time to adequately develop the maps, were a common theme. One wrote that some 
maps could be used more easily for science concepts than others. 

The students of teachers who reported that they used Thinking Maps® regularly and 
consistently scored higher than those students of the teachers who did not have a positive 
perception of or who did not use Thinking Maps® regularly in their classes. 


Research Question 3 

What, if any, is the relationship between the use of Thinking Maps® and teachers’ 
perception and implementation in middle grades? 

Based on the results from statistical testing in SPSS and SAS, and perceptions of science 
teachers who felt favorably towards using Thinking Maps® regularly in their science classes, it 
can be concluded there is a trend in the perception and consistent usage of Thinking Map® in the 
middle grades science classes and the increase in achievement score means of the students who 
use them regularly. Additionally, because an interaction effect cannot be calculated due to the 
variance in the data to report a relationship, a comparison of the means for each grade level and 
non-usage or usage of Thinking Maps® showed that in four out of six instances (67%) that 
students who used Thinking Maps® scored a higher mean than students who did not use 
Thinking Maps®. 


ANALYSIS OF RESEARCH FINDINGS 

Data analysis of the Georgia CRCT science test scores was completed with SPSS 24 and 

SAS. For the 2012-2013 data, non-parametric statistical testing indicated that grade level had an 
effect on the scores, and that the seventh grade had the greatest variance in scores. Further 



testing for non-usage and usage indicated that usage of Thinking Maps® did not have an 
influence on the CRCT scores to a statistically significant degree. Friedman’s non-parametric 
ANOVA was used to determine the influence of both grade level and amount of usage on the 
CRCT science scores. The model was significant, revealing that grade level and usage of 
Thinking Maps® together explained 9.11% of the variance in the CRCT scores. However, 
because the scores followed a non-normal distribution, and the CRCT scores did not originate 
from the same distribution, it was difficult to determine if the interaction between grade levels 
and Thinking Maps® were significant. 

For the 2013-2014 data, non-parametric statistical testing indicated that grade level had 
an effect on the scores to a statistically significant degree. Again, the seventh grade had the 
greatest variance in scores. Testing for non-usage and usage of Thinking Maps® showed that 
usage of Thinking Maps® had an influence on the CRCT scores. Friedman’s non-parametric 
ANOVA was used to determine the simultaneous influence of both grade level and amount of 
usage on the CRCT scores. The model was significant, indicating that grade level and usage 
together explained 23.2% of the variance in the CRCT scores. Again, because of the non-normal 
distribution of the data, and because the CRCT scores did not originate from the same 
distribution, an interaction effect was difficult to interpret. 

DISCUSSION 

Based on results from non-parametric statistical testing, the 2012-2013 and 2013-2014 
data showed that grade level had an influence on CRCT scores in science, and that the 
combination of grade level and usage explained 9.11% and 23.2% of the variance in the scores, 
respectively. Additionally, the usage of Thinking Maps® had an effect on the CRCT scores as 


related to the 2013-2014 data. 
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In a comparison of these findings with the review of literature, it was revealed that of the 
thirteen studies conducted on the use of Thinking Maps®, including advance organizers or other 
graphic organizers, eight of the thirteen, or 62%, focused on elementary students, three of the 
thirteen, or 23%, focused on college students, and only two of the thirteen, or 15%, focused on 
middle school students (see Concept Analysis Chart, p. 39). Only one of the two studies 
reported included quantitative data regarding achievement test scores. Ausubel (1960) used 
advanced organizers with college students to determine if learning could be facilitated, which 
was shown by an increase in assessment scores. Ball (1998) used Thinking Maps® to improve 
reading scores, with 5 of 7 variables showing improvement, and Gallagher (2011) using 
Thinking Maps® when generating writing about dietary articles. Articles written by the students 
were better organized and the clarity of the writing improved. 

At the elementary level, Hyerle (2000) used Thinking Maps®, showing gains in reading, 
writing, and math without reporting the data he collected. Chang, et al., (2002) revealed that 
concept mapping aided fifth graders in knowledge acquisition. Brown (2003) tested to see if 
fourth graders could develop higher-order comprehension tests for their peers using Thinking 
Maps®. Hickie (2006) found there was a difference in using Thinking Maps® in reading and 
language arts with fifth graders, and Gibbs (2009) showed that use of Thinking Maps® 
influenced reading scores with third and fourth graders. Leary (1999), Russell (2010), and 
Sunseri (2011) reported no difference in the achievement scores of students who used Thinking 
Maps® over those who did not use Thinking Maps®. 

Of the two studies at the middle grades level, Hester, et al., (1995) reported an “upswing” 
in achievement test scores and “outstanding” growth in reading and writing, but did not report 



any data. DiCecco and Gleason (2002) reported that using graphic organizers helped increase 
achievement test scores of learning disabled (LD) students over those who did not. 

Based on the review of literature, therefore, it is difficult to make a comparison when 
either the data is missing with which to compare it or the program being tested does not support 
the dependent variable. Because Hester, et al., did not report the actual data collected, and 
because DiCecco and Gleason were using graphic organizers and not true Thinking Maps®, the 
data regarding Thinking Maps® is lacking for the middle grades and for science. This study, 
then, is important to the literature in that it presents data collected at the middle grades level and 
for the science content area using Thinking Maps® and not another program. 

CONCLUSIONS 

This study sought to determine if there was an effect of the usage of the Thinking Maps® 
program on middle grades’ students’ science achievement test scores on the 2012-2013 and 
2013-2014 administrations of the Georgia CRCTs. The primary question that was raised in the 
study was if there was an effect in achievement as measured by the state-mandated content area 
tests for three grade levels. The results from data analysis concluded that grade level had an 
effect on the achievement scores and that the combination of grade level and usage explained 
9.11% and 23.2%, respectively, of the variance in the scores over the two years of the study. 
However, usage did not have a statistically significant effect during the 2012-2013 study year, 
but did show a statistically significant effect during the 2013-2014 study year. Studies 
completed by DiCecco and Gleason (2002) and Hester, et al., (1995) were they only studies at 
the middle grades level. DiCecco and Gleason’s study focused on graphic organizers and not 
specifically Thinking Maps®, and Hester’s study included results but was not supported by a 


reporting of the data. 
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The second question examined teachers’ perceptions of the Thinking Maps® program, 
seeking to reveal if a positive perception by the teacher might have a secondary effect on the 
achievement scores. Eighty-six percent of the respondents said they believed Thinking Maps® 
help students learn science content, but several did not use them due either to a dislike of the 
program or because of the challenges inherent in using them, such as using a variety or maps and 
a lack of time for the students to practically develop them. The students of those teachers who 
regularly used Thinking Maps® had a higher mean than those who did not. 

The final question sought to unify the two previous questions to see if not only the usage 
of Thinking Maps® affected achievement test scores, but that the science teachers’ perceptions 
regarding Thinking Maps® affected achievement test scores simultaneously. Based on the 
statistical testing and the positive perceptions of over 80% of the science teachers, it can be 
concluded that there is a trend in the simultaneous positive perception and consistent usage of 
Thinking Maps® in the middle grades science classes and the increase in achievement score 
means of students who use them consistently and with fidelity. 

Based on the review of literature and the findings from statistical testing and teacher 
perceptions, it may be concluded that Thinking Maps® have an effect on the achievement scores 
of middle grades students in science when used consistently and with fidelity. The number of 
previous studies at different levels attributes to this conclusion, starting with Ausubel (1960), 
Ball (1998), and Gallagher (2011) having positive results at the collegiate level. At the 
elementary level, Chang, et al., (2002) saw results with concept mapping, and Gibbs (2009) and 
Hickie (2006) saw results with Thinking Maps®. Others, including Brown (2003) and Hyerle 
(2000) discussed positive results with Thinking Maps®, while Leary (1999), Russell (2010), and 
Sunseri (2011) reported no effect of Thinking Maps® on achievement. The researcher has 



witnessed increased student achievement, not only in the classroom, but on formal and informal 


assessments when Thinking Maps® and other visual-verbal organizers were used with 
consistency. The ability to organize and manipulate information appears to be the deciding 
factor. 


RELATIONSHIP TO RESEARCH 

In a comparison of these findings with the review of literature, it was revealed that of the 
thirteen studies completed using Thinking Maps®, including advance organizers or other graphic 
organizers, eight of the thirteen, or 62%, focused on elementary students, three of the thirteen, or 
23%, focused on college students, and only two of the thirteen, or 15%, focused on middle 
school students (see Concept Analysis Chart, p. 39). Only one of the two studies at the middle 
school level reported included quantitative data regarding achievement test scores. Ausubel 
(1960) used advanced organizers with college students to determine if learning could be 
facilitated, which was shown by an increase in assessment scores. Ball (1998) used Thinking 
Maps® to improve reading scores, with 5 of 7 variables showing improvement, and Gallagher 
(2011) using Thinking Maps® when generating writing about dietary articles. Articles written 
by the students were better organized and the clarity of the writing improved. 

At the elementary level, Hyerle (2000) used Thinking Maps®, showing gains in reading, 
writing, and math without reporting the data he collected. Chang, et al., (2002) revealed that 
concept mapping aided fifth graders in knowledge acquisition. Brown (2003) tested to see if 
fourth graders could develop higher-order comprehension tests for their peers using Thinking 
Maps®. Hickie (2006) found there was a difference in using Thinking Maps® in reading and 
language arts with fifth graders, and Gibbs (2009) showed that use of Thinking Maps® 
influenced reading scores with third and fourth graders. Leary (1999), Russell (2010), and 
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Sunseri (2011) reported no difference in the achievement scores of students who used Thinking 
Maps® over those who did not use Thinking Maps®. 

Of the two studies at the middle grades level, Hester, et al., (1995) reported an “upswing” 
in achievement test scores and “outstanding” growth in reading and writing, but did not report 
any data. DiCecco and Gleason (2002) reported that using graphic organizers helped increase 
achievement test scores of learning disabled (LD) students over those who did not. 

Based on the review of literature, therefore, it is difficult to make a comparison when 
either the data is missing with which to compare it or the dependent variable is not supported by 
the program being tested. Because Hester, et al., did not report the actual data collected, and 
because DiCecco and Gleason were using graphic organizers and not true Thinking Maps®, the 
data regarding Thinking Maps® is lacking for the middle grades and for science. This study, 
then, is important to the literature in that it presents data collected at the middle grades level and 
for the science content area using Thinking Maps® and not another program. 

IMPLICATIONS FOR PRACTICE 

Educators are constantly on the lookout for the next program to help their students learn 
their content area curriculum. They rely on researchers to have tested the program for success 
based on the claims made by the developers. Since there is a lack of evidentiary support for 
student achievement at the middle grades level, there must be research and testing to determine if 
the program can live up to the claims, that is, can achievement scores at the middle school level 
be increased by using this program? If the program does not live up to its claim, then the money 
spent on it is wasted. Therefore, it is necessary to complete statistical testing so that the funds 


are used to the benefit of the students. 



Achieving in science at the middle school level, given the challenges facing middle 
schoolers, is a short-term implication. The long-term implication is that the students learn how 
to organize and utilize information for better processing and retention, which is a life skill. This 
achievement links the short-term to the long-term. If the usage of Thinking Maps® helps 
students, at any level, learn to organize and use information for a variety of purposes, then it is 
worth the price paid for it. However, if it does not help students learn, it should not be 
continued, and other programs should be explored for their ability to help students learn science 
content, or any other content. Using a program that does not work is analogous to pedaling 
furiously on a bicycle bolted to the floor. You can pedal all you want, but you will not get 
anywhere. A student using a program that makes no discernible difference in his ability to learn 
instructional content or excel on assessments is being deprived of his ability to leam, grow, and 
succeed. 


RECOMMENDATIONS 

Based on the findings of this research study, the first recommendation would be to 
continue studying the Thinking Maps® program, not only at the middle grades level, but at the 
other levels as well. Knowing if the program does what it is purported to do will better ensure 
that students receive the most benefit. This program, while having some data on which to fall 
back, needs the benefit of more study. Along that line, it is also recommended to replicate this 
study in districts with different demographics from the selected school within five years. 

Another recommendation to the developers and trainers of Thinking Maps® would be to 
provide more examples and templates of the maps to prospective users of the program for 
content areas. A complaint among the science teachers on the teacher survey was that teachers 


had a difficult time planning or finding a variety of maps for their instructional objectives in their 
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content area. Providing more examples and templates for those content areas would help to 
foster usage among the educators, helping teachers see Thinking Maps® in a more positive light, 
and ultimately helping the students learn those instructional objectives. 

A final recommendation would be to those researchers who completed formal and 
informal testing or studies of the use of Thinking Maps® and yet did not report those findings. It 
does the program under scrutiny no good service if you report the findings and then do not back 
it up with the quantitative (or qualitative) data results. 

DISSEMINATION 

Sharing information is crucial to educators. When teachers find something that works, or 
does not work, it is important for them to share that information with their colleagues. Prior to 
getting approval for the study, the researcher was interviewed by the assistant superintendent for 
instruction and a plan was formulated to present the results of the study to her and to the faculty 
of the four middle schools at a professional development seminar. This also includes the faculty 
at the selected school. Additionally, an article for the Middle School Journal , a refereed journal 
published by the Association of Middle Level Education (AMLE) will be prepared and 
submitted for publication. 
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APPENDIX A 

Survey Questions for Teachers on Perceptions and Use of Thinking Maps® 


Thinking Maps® Teacher Perception Survey 

Our goal is to find out your beliefs, views, or perceptions about Thinking Maps®. 

Please do not write your name on this survey. Please answer each question honestly. Your answers will be 
confidential. 


Place a check or X beside your answer choice for each category. 

Age 21-30 31-40 41-50 51-60 61+ 
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Gender Female 


Male 

Grade _6 

Years 

Teaching 0-5 6-10 

11-15 

16-20 21-25 

26-30 

1. In which science subject 
area do you primarily use 

Thinking Maps®? 


_Earth & Space _ 

_Life _Physical 


Choose yes or no by placing a check or an X beside your answer choice for each question. 

2. Did you receive training in Thinking Maps® in 2008-2009? yes no 


3. If no, did you receive formal training in Thinking Maps®? yes no 


4. Do you believe Thinking Maps® help students learn science content? yes no 


Please order from 1-3 as you choose which Thinking Maps® best answers each question. 


Thinking Maps® 

Question 

Brace 

Bridge 

Bubble 

Circle 

Double 

Bubble 

Flow 

Multi- 

Flow 

Tree 

5. In your opinion, which 3 
Thinking Maps® do you 
believe are most effective? 









6. In your opinion, which 3 
Thinking Maps® do you 
believe are least effective? 









7. Which 3 Thinking Maps® 
do you utilize the most? 1 = 
used most 










Circle the number that best matches the description of your answer to the question. 


8. How often do you use Thinking Maps®? 

1 2 3 

Every 3x a 2x a 

day week week 


9. How often do you use Thinking Maps® with a class? 


1 


2 


6-7x a 
day 


4-5x 

a 

day 


3 

2-3x a 
day 


4 

Once a 
week 


4 

Once a 
day 


5 

Never 


5 

Never 


Please answer each question and explain clearly. 

10. Describe how you typically use Thinking Maps® in your classes and please give an example. 






11. Asa professional educator, do you believe Thinking Maps® achieve the outcome for which they were 
designed, to help students organize, manipulate, and learn information? Please explain and give examples that 
support your response. 


12. What has been the most challenging aspect of using Thinking Maps® in your daily practice? 


Thank you for completing our survey! 




